Myopic and nonmyopic agent optimization in game theory, economics, biology and artificial intelligen

1
Myopic and non-myopic agent optimization in game
theory, economics, biology and artificial
intelligence

• Michael J Gagen
• Institute of Molecular Bioscience
• University of Queensland
• Email m.gagen_at_imb.uq.edu.au

Kae Nemoto Quantum Information Science National
Institute of Informatics, Japan Email
nemoto_at_nii.ac.jp
2
Overview Functional Optimization in Strategic
Economics (and AI)
? Formalized by von Neumann and Morgenstern,
Theory of Games and Economic Behavior (1944)
Mathematics / Physics (minimize action)
3
Overview Functional Optimization in Strategic
Economics (and AI)
? Formalized by von Neumann and Morgenstern,
Theory of Games and Economic Behavior (1944)
Strategic Economics (maximize expected payoff)
Functionals Fully general Not necessarily
continuous Not necessarily differentiable
Nb Implicit Assumption of Continuity !!
4
Overview Functional Optimization in Strategic
Economics (and AI)
Strategic Economics (maximize expected payoff)
von Neumanns myopic assumption
Evidence von Neumann Nash used fixed point
theorems in probability simplex equivalent to a
convex subset of a real vector space
von Neumann and Morgenstern, Theory of Games and
Economic Behavior (1944) J. F. Nash, Equilibrium
points in n-person games. PNAS, 36(1)4849 (1950)
5
Overview Functional Optimization in Strategic
Economics (and AI)
Non-myopic Optimization
No communications between players
Correlations ? Constraints and forbidden regions
6
Overview Functional Optimization in Strategic
Economics (and AI)
X
Myopic ? The Game Tree lists All play options
7
Myopic Missing Information!
Correlation Information
What Information?
Nemoto It is not what they are doing, its what
they are thinking!

• Chess
• Chunking or pattern recognition in human chess
  play
• Experts
• Performance in speed chess doesnt degrade much
• Rapidly direct attention to good moves
• Assess less than 100 board positions per move
• Eye movements fixate only on important positions
• Re-produce game positions after brief exposure
  better than novices, but random positions only
  as well as novices
• Learning Strategy Learning information to help
  win game

8
Optimization and Correlations are Non-Commuting!
Complex Systems Theory Emergence of Complexity
via correlated signals ? higher order structure
9
Optimization and Correlations are Non-Commuting!
Life Sciences (Evolutionary Optimization) Selfish
Gene Theory Mayr Incompatibility between biology
and physicsRosen Correlated Components in
biology, rather than uncorrelated
partsMattick Biology informs information
science 6 Gbit DNA program more complex than
any human program, implicating RNA as
correlating signals allowing multi-tasking and
developmental control of complex organisms.
10
Optimization and Correlations are Non-Commuting!
Economics Selfish independent agents homo
economicus Challenges Japanese Development
Economics, Toyota Just-In-Time Production
System
11
Optimization and Correlations are Non-Commuting!
1 Player Evolving / Learning Machines (neural and
molecular networks) endogenously exploit
correlations to alter own decision tree, dynamics
and optima
o F(i) F(t,d) Ft (d) ? F(x,y,z),
,F(x,x,z), Functional Programming, Dataflow
computation, re-write architectures,
12
Discrepancies Myopic Agent Optimization and
Observation
Heuristic statistics
13
Myopic Agent Optimization
Normal Form
Strategic Form
Px
Py
Sum-Over-Histories or Path Integral formulation
von Neumann and Morgenstern (1944) All possible
information All possible move combinations for
all histories and all futures
14
Myopic Agent Optimization
15
Myopic Agent Optimization
16
Non-Myopic Agent Optimization
17
Non-Myopic Agent Optimization in the Iterated
Prisoners Dilemma
In 1950 Melvin Dresher and Merrill Flood devised
a game later called the Prisoners Dilemma Two
prisoners are in separate cells and must decide
to cooperate or defect Cooperation Defect CKR
Common Knowledge of Rationality
Payoff Matrix
18
Non-Myopic Agent Optimization in the Iterated
Prisoners Dilemma
Myopic agent assumption
19
Non-Myopic Agent Optimization in the Iterated
Prisoners Dilemma
Myopic agents N max constraints
Simultaneous solution ? Backwards Induction ?
myopic agents always defect
20
Non-Myopic Agent Optimization in the Iterated
Prisoners Dilemma
Correlated Constraints (deriving Tit For Tat)
lt 0 ? P1x(1) 0, so Px cooperates lt 0 ? P1y(1)
0, so Py cooperates
21
Non-Myopic Agent Optimization in the Iterated
Prisoners Dilemma
Families of correlation constraints k, j index
Change of notation dot N N, dot dot N
2N, dot dot N-2 2N-2, etc
Optimize via game theory techniques Many
constrained equilibria involving
cooperation Cooperation is rational in IPD
22
Further Reading and Contacts
Kae Nemoto Email nemoto_at_nii.ac.jp URL http//ww
w.qis.ex.nii.ac.jp/knemoto.html
Michael J Gagen Email m.gagen_at_imb.uq.edu.au URL
http//research.imb.uq.edu.au/m.gagen/
See Cooperative equilibria in the finite
iterated prisoner's dilemma, K. Nemoto and M. J.
Gagen, EconPaperswpawuwpga/0404001
(http//econpapers.hhs.se/paper/wpawuwpga/0404001.
htm)
23
(No Transcript)