Falcon on a Cloudy Day

1
Falcon on a Cloudy Day

• A Ro Sham Bo Algorithm
• by Andrew Post

2
Lets Review

• If you missed my previous presentation
• Ro Sham Bo Rock Paper Scissors
• Can be more complicated though
• Ro Sham Bo has important applications
• Algorithms compete at Ro Sham Bo in tournaments
• Iocaine Powder is the world champ of Ro Sham Bo
• Because it uses Sicilian Reasoning
• I will beat Iocaine Powder
• Eventually

3
What is Ro Sham Bo?

• Also known as Rock Paper Scissors

4
What is Ro Sham Bo?

• Generalized case of Rock Paper Scissors actually
• Not always three choices
• Ties can be resolved differently
• The game is not necessarily zero-sum

5
(No Transcript)
6
Why does it matter?

• Many competitive scenarios involve a Ro Sham Bo
• Example
• CBS and NBC choosing Primetime TV Shows
• They can choose to show a Drama, Comedy, or
  Sports show
• Viewers prefer Comedy to Drama, Sports to Comedy,
  and Drama to Sports, given the choice.
• Neither station knows ahead of time what the
  other will choose
• Billions of dollars every day rely on decisions
  like these.

7
How it works

• Simplest Non-Cooperative Game
• Players cannot play to ensure they both win
• Governed by the Nash Equilibrium
• There are strategies which cannot be dominated
• http//www.youtube.com/watch?vpdrBDfRvpBA 131
  -- 220

8
How to Win

• As you just heard, playing randomly can ensure
  you dont lose, but how do you win?
• How to predict your opponent
• Sub-Optimal Frequency Distributions
• Pattern Matching
• History Analysis

9
Iocaine Powder

• International Ro Sham Bo Programming Tournament
  Champion
• Named for this famous scenehttp//youtube.com/wa
  tch?vTUee1WvtQZU057 -- 220

10
The Tournament

• Tournament programs play thousands of rounds
• Win by beating the most opponents by a large
  margin
• Most programs play sub-optimally, so exploiting
  your opponent is more important than playing
  randomly to avoid losing.

11
(No Transcript)
12
Iocaine Powder

• IP is the algorithm which does this best.
• IP uses the same heuristics to predict what an
  opponent is most likely to do.
• Using the same tools, how can you be better?
• Sicilian Reasoning!

13
Sicilian Reasoning

• Levels of second guessing
• Opponent will play rock, so play paper
• Opponent knows you will counter rock with paper,
  and play scissors so play rock
• Opponent knows all this, and will now play paper
  to beat your rock so play scissors
• Opponent will play rock again same as 1

14
Sicilian Reasoning

• Use your predictive strategies to evaluate what
  is going to happen next.
• Run SR on yourself and your opponent, and keep a
  table of what each of the six levels of reasoning
  say you should do.
• Pick the level of reasoning which would have won
  against what your opponent actually did the most
  often.

15
Wait, six? Dont you mean three?

• You can use the same predictive tools that your
  opponent uses to predict what you are going to
  do.
• Now you have three more levels of SR
• 4. I will play rock. So he plays paper. So play
  Scissors
• 5. He knows I will counter with scissors, and
  play rock. So play Paper.
• 6. He expects me to counter-counter with paper,
  and will play scissors. So play rock.

16
More Sicilian Reasoning

• Just because one level of SR is winning now,
  doesnt mean it always will be.
• Opponents will change how they play if they are
  losing, so you must change too!
• How do you switch your level of SR?

17
Switching Reasoning

• SR-2 has just won the first 100 rounds
• Opponent changes strategy
• You lose 50 rounds before SR-4 has more than 100
  theoretical wins.
• You just wasted 50 rounds!

18
Switching Reasoning

• Use several different methodologies for switches
• Most wins in last 10, 25, 50, 100, 1000 rounds
• Has won the most in similar situations
• Causes the opponent to switch to a worse strategy

19
Switching Reasoning

• Here is the real genius now use the switching
  methodology which has helped you win the most
  rounds!

20
Falcon on a Cloudy Day

• So you ask, how do you beat Iocaine Powder?
• Improve the basic predictive heuristics
• Extend Sicilian Reasoning

21
Improving Prediction

• What I have implemented
• Improved Variable History Analysis
• Look at just your history, your opponents, or
  both
• Improved Frequency Analysis
• EVx Prx2 - Prx1

22
Demonstration

• Here is how my project does with what is
  implemented so far.

23
Improving Prediction

• What I have not implemented yet
• Improved Pattern Matching
• Markov Models with MegaHAL
• Extended Sicilian Reasoning

24
More on MegaHAL

• MegaHAL is a very simple "infinite-order" Markov
  model.
• Stores frequency information about the moves the
  opponent has made in the past for all possible
  contexts
• Using the context of the last few moves, the
  appropriate response is then selected.

25
Extended Sicilian Reasoning

• Q Isnt Sicilian Reasoning complete at 6?
• A Yes, but there is information we are ignoring.
• By compressing your strategy decisions into the
  idea of which of six strategies is best right
  now, you have no way to keep track of how
  changing your strategies has paid off best in the
  past.

26
Now for some Math

• Hilbert Space
• Game Trajectory and Game State
• Projection Operators
• Annotated History Analysis
• Project Enigma