Towards Realistic Models for Evolution of Cooperation

1
Towards Realistic Models for Evolution of
Cooperation

• LIK MUI

2
about procedure

• Briefly go over the paper
• Clarify major points
• Describe simulations (not in paper)

3
RoadMap

• Introduction
• Cooperation Models
• Simulations
• Conclusion

?
4
Evolution of Cooperation

• Animals cooperate
• Two questions
• How does cooperation as a strategy becomes stable
  evolutionarily?
• How does cooperation arise in the first place?

5
Darwinian Natural Selection

• Survival of the fittest
• If evolution is all about individual survival,
  how can cooperation be explained?
• Fittest what?

6
Fittest what ?

• Individual
• Rational agency theory (Kreps, 1990)
• Group
• Group selection theory (Wilson, 1980)
• Gene
• Selfish gene hypothesis (Dawkins, 1979)
• Organization
• Classic organizational theory (Simon, 1969)

7
RoadMap

• Introduction
• Cooperation Models
• Group Selection
• Kinship Theory
• Direct Reciprocity
• Indirect Reciprocity
• Social Learning
• Simulations
• Conclusion

?
8
Group Selection

• Intuition we ban cannibalism but not
  carnivorousness
• Population/species basic unit of natural
  selection
• Problem explain war, family feud, competition,
  etc.

9
Kinship Theory I

• Intuition nepotism
• Hamiltons Rule
• Individuals show less aggression and more
  cooperation towards closer kin if rule is
  satisfied
• Basis for most work on kinship theory
• Wrights Coefficient of Related r
• Self r1
• Siblings r0.5
• Grandparent-grandchild r0.25

10
Kinship Theory II

• Cannot explain
• Competition in viscuous population
• Symbioses
• Dynamics of cooperation

11
Direct Reciprocity

• Intuition being nice to others who are nice
• Reciprocal Altruism
• Trivers (1971)
• Tit-for-tat and PD tournament
• Axelrod and Hamilton (1981)
• Cannot explain
• We cooperate not only with people who cooperate
  with us

12
Indirect Reciprocity

• Intuition respect one who is famous
• Social-biological justifications
• Biology generalized altruism (Trivers, 1971,
  1985)
• Sociobiology Alexandar (1986)
• Sociology Ostrom (1998)
• 3 types of indirect reciprocity
• Looped
• Observer-based
• Image-based

13
Indirect Reciprocity Looped

• Looped Indirect Reciprocity
• Boyd and Richerson (1989)

14
Indirect Reciprocity Observers

• Observer-based Reciprocity
• Pollock and Dugatkin (1992)

15
Indirect Reciprocity Image

• Image (reputation) based Reciprocity
• Nowak and Sigmund (1998, 2000)

16
Social Learning

• Intuition imitate those who are successful
• Cultural transmission
• Boyd and Richerson (1982)
• Docility
• Simon (1990, 1991)

17
Critiques of Existing Models

• Many theories each explaining one or a few
  aspects of cooperation
• Unrealism of model assumptions

18
Unrealism for Existing Models

• asexual, non-overlapping generations
• simultaneous play for every interaction
• c.f., Abell and Reyniers, 2000
• dyadic interactions
• mostly predetermined behavior
• c.f., May, 1987 (lack of modeling stochasticity)
• discrete actions (cooperate or defect)
• social structure and cooperation
• c.f., Simon, 1991 Cohen, et al., 2001
• extend social learning
• c.f., Simon, 1990

19
RoadMap

• Introduction
• Cooperation Models
• Simulations
• Nowak and Sigmund Game
• Prisoners Dilemma Game
• Simons Docility Hypothesis
• Conclusion

?
20
Nowak and Sigmund Game

• Payoff Matrix
• C 0.1
• B 1.0
• Image Adjustment
• A 1

Interact Not interact
Donor -C 0
Recipient B 0
Interact Not interact
Donor A -A
Recipient 0 0
21
Using Global Image 1 Run
22
Using Global Image 100 Runs
23
Dynamics using Global Reputation
24
Using 10 Observers/Interactions
25
Evolutionary PD Game

• Repeated Prisoners Dilemma Game
• Agent Actions
• Action cooperate, defect
• Payoff Matrix

C D
C 3/3 0/5
D 5/0 1/1
26
PD Game Agent Strategies

• All defecting (AllD)
• Tit-for-tat (TFT)
• Reputational Tit-for-tat (RTFT) using various
  notions of reputation

27
Base Case PD Game
28
Simple Groups social structures

• Group structure affects members
• Interactions, observations, and knowledge
• Persistent structure
• Groups actions
• Observed indirectly through member's actions

29
Group Membership

• Member agents
• Have public group identity
• Directly associated with one environment
• Group Structure is a Tree
• Least common ancestral node (LCAN)
• Events occur with respect to a shared environment

30
Shared Environment Example

• Agents Group
• A1,A2 G1
• A3,A4 G2
• A5,A2 G1
• A1,A3 G0
• A5,A3 G0

31
PD Game with Group Reputation(varying encounters
per generation EPG)
32
PD Game with Group Reputation (100 EPG varying
Inter-group interaction probability)
33
Groups/Organizations bounded rationality
explanation

• Docility
• Cooperation (altruism) as an explanation for the
  formation of groups/organizations
• Why individuals identify with a group?
• boundedly rational individuals
• increase their survival fitness
• (Simon, 1969, 1990, 1991)

34
PD Game with Docility(50 cooperators and 50
defectors 100 EPG 1.0 IP)
35
Conclusion

• Reviewed 5 major approaches to study evolution of
  cooperation
• Provided 2 main critiques for existing models
• Constructed model extensions addressing the
  critiques

36
Implications for Computer Science

• Artificial intelligence
• Benevolent agents are not good enough
• (c.f., multi-agents systems)
• Learning theory can be used to study evolution of
  cooperation
• Systems
• Improve system design by understanding the
  dynamics of agents
• Accountability substrate needed for distributed
  systems

37
Future Plan

• Extend the simple group social structure
• Overlapping generations
• Sexual reproduction
• Extend social learning using realistic/robust
  learning model

38
Modeling Diploid Organisms
39
Modeling Diploid Organisms
40
Modeling Diploid Organisms
One of 2 Child Chromosomes
Parental Chromosomes
41
Simulation Demo
C D
C R/R S/T
D T/S P/P

• Recall PD payoff matrix

• PD strategies viewed as a probability vectors
• Strategy ltPI, PT, PR, PP, PSgt
• TFT lt 1, 1, 1, 0, 0 gt
• AllD lt 0, 0, 0, 0, 0 gt
• AllC lt 1, 1, 1, 1, 1 gt
• STFT lt 0, 1, 1, 0, 0 gt

42
Simulation a search problem

• Search Optimal PD Strategy
• Search space I, T, R, P, S probabilities