Finite State Machine for Games

1
Finite State Machine for Games

• Spring 2005
• Ref Chenney, CS679 lectures
• AI Game Programming Wisdom 2

2
Outline

• AI and Game
• Introduction/examples
• Design
• Intuition
• State-based
• Implementation
• Extending
• Stack-based
• Fuzzy-state machine

3
What is AI?

• AI is the control of every non-human entity in a
  game
• The other cars in a car game
• The opponents and monsters in a shooter
• Your units, your enemys units and your enemy in
  a RTS game
• But, typically does not refer to passive things
  that just react to the player and never initiate
  action
• Thats physics or game logic
• For example, the blocks in Tetris are not AI, nor
  is a flag blowing in the wind

4
AI in the Game Loop

• AI is updated as part of the game loop, after
  user input, and before rendering
• There are issues here
• Which AI goes first?
• Does the AI run on every frame? (LOD problem)
• Is the AI synchronized?

5
AI and Animation

• AI determines what to do and the animation does
  it
• AI drives animation, deciding what action the
  animation system should be animating
• Scenario 1 The AI issues orders like move from
  A to B, and its up to the animation system to
  do the rest
• Scenario 2 The AI controls everything down to
  the animation clip to play
• Which scenario is best depends on the nature of
  the AI system and the nature of the animation
  system
• Is the animation system based on move trees
  (motion capture), or physics, or something else
• Does the AI look after collision avoidance? Does
  it do detailed planning?

6
AI Update Step

• The sensing phase determines the state of the
  world
• May be very simple - state changes all come by
  message
• Or complex - figure out what is visible, where
  your team is, etc
• The thinking phase decides what to do given the
  world
• The core of AI
• The acting phase tells the animation what to do
• Generally not interesting

AI Module
Sensing
Game Engine
Thinking
Acting
7
AI by Polling

• The AI gets called at a fixed rate
• Senses It looks to see what has changed in the
  world. For instance
• Queries what it can see
• Checks to see if its animation has finished
  running
• And then acts on it
• Why is this generally inefficient?
• Different characters might require different
  polling rate

8
Event Driven AI

• Event driven AI does everything in response to
  events in the world
• Events sent by message (basically, a function
  gets called when a message arrives, just like a
  user interface)
• Example messages
• A certain amount of time has passed, so update
  yourself
• You have heard a sound
• Someone has entered your field of view
• Note that messages can completely replace
  sensing, but typically do not. Why not?
• Real system are a mix - something changes, so you
  do some sensing

9
AI Techniques in Games

• Basic problem Given the state of the world, what
  should I do?
• A wide range of solutions in games
• Finite state machines, Decision trees, Rule based
  systems, Neural networks, Fuzzy logic
• A wider range of solutions in the academic world
• Complex planning systems, logic programming,
  genetic algorithms, Bayes-nets
• Typically, too slow for games

10
Goals of Game AI

• Several goals
• Goal driven - the AI decides what it should do,
  and then figures out how to do it
• Reactive - the AI responds immediately to changes
  in the world
• Knowledge intensive - the AI knows a lot about
  the world and how it behaves, and embodies
  knowledge in its own behavior
• Characteristic - Embodies a believable,
  consistent character
• Fast and easy development
• Low CPU and memory usage
• These conflict in almost every way

11
Two Measures of Complexity

• Complexity of Execution
• How fast does it run as more knowledge is added?
• How much memory is required as more knowledge is
  added?
• Determines the run-time cost of the AI
• Complexity of Specification
• How hard is it to write the code?
• As more knowledge is added, how much more code
  needs to be added?
• Determines the development cost, and risk

12
Expressiveness

• What behaviors can easily be defined, or defined
  at all?
• Propositional logic
• Statements about specific objects in the world
  no variables
• Jim is in room7, Jim has the rocket launcher, the
  rocket launcher does splash damage
• Go to room8 if you are in room7 through door14
• Predicate Logic
• Allows general statement using variables
• All rooms have doors
• All splash damage weapons can be used around
  corners
• All rocket launchers do splash damage
• Go to a room connected to the current room

13
Finite State Machines (FSMs)

• A set of states that the agent can be in
• Connected by transitions that are triggered by a
  change in the world
• Normally represented as a directed graph, with
  the edges labeled with the transition event
• Ubiquitous in computer game AI
• You might have seen them, a long time ago, in
  formal language theory (or compilers)
• What type of languages can be represented by
  finite state machines?
• How might this impact a characters AI?
• How does it impact the size of the machine?

14
Formal Definitions (N. Philips)

• "An abstract machine consisting of a set of
  states (including the initial state), a set of
  input events, a set of output events, and a state
  transition function.
• The function takes the current state and an input
  event and returns the new set of output events
  and the next state. Some states may be designated
  as "terminal states".
• The state machine can also be viewed as a
  function which maps an ordered sequence of input
  events into a corresponding sequence of (sets of)
  output events.
• Finite State Automaton the machine with no
  output

15
FSM with Output vending machines

• description
• State table

16
Vending Machine state diagram
17
FSM and Game

• Game character behavior can be modeled (in most
  cases) as a sequence of different mental state,
  where change is driven by the actions of
  player/other characters,
• Natural choice for defining AI in games

18
FSM with No Output
19
Ex predator vs. prey

• Prey (laalaa)

Idle (stand,wave,)
Sees predator
Flee (run)
No more threat
captured
20
Predator (Raptor)
Idle (stand)
Tidle gt 5
Hungry (wander)
Tdininggt5
Dining
Prey captured
Prey in sight
Pursuit (run)
Tpursuit gt 10
21
Idling LaaLaa
This page illustrates hierarchical
state, Non-deterministic state transition
Target arrived
Wander (set random target)
20
Stand
Tstandgt4
30
R
Wave
50
Twavegt2
22
(No Transcript)
23
FSM Cinematography

• The current state (and state transition) may have
  an important effect on how camera should be
  manipulated
• Idling focus on character
• Movement zoom out to show spatial info
• State-transition camera animation
• More complicated camera behavior may be coded in
  another FSM

24
Camera Selection Strategy
It is important to choose a camera that has the
front of the character
X
Z
25
Camera Selection (cont)
A good camera -v lies in the frustum
-v
26
Other Concerns

• Adjust zoom (fovy) so that the character occupies
  a good (fixed!?) portion of the screen

d
near
27
FOVY based on Postures

• Once in running mode, zoom out
• In stand/wave modes, zoom in
• Smooth transition between two fovy settings (by
  indirectly controlling the percent value)

28
FSM Design
29
Quake2 Examples
Intuitive thinking model the events and state
changes
Quake2 uses 9 different states standing,
walking, running, dodging, attacking, melee,
seeing the enemy, idle and searching.
Incomplete design
30
Quake Rocket
31
Shambler monster
32
Intuitive Design

• Say, a simple teletube baby has three states
  idle, run, and wave
• Scenario
• When an idle laalaa sees a butterfly, it waves to
  it. When the butterfly flies away, it returns to
  idle
• When an idle laalaa sees a mouse, it flees away.
  When the mouse is no longer in sight, it returns
  to idle

33
Laalaa
flee
mouse
mouse
How to make sure the design complete? I.e., all
states and transitions are considered Is there
any systematic way of developing an FSM?
idle
butterfly
wave
butterfly
34
Quake Bot Example (first-cut)

• Types of behavior to capture
• Wander randomly if dont see or hear an enemy
• When see enemy, attack
• When not see enemy and hear an enemy, chase enemy
• When die, re-spawn (new a bot from start)
• Events see enemy, hear enemy, die
• States wander, attack, chase, spawn

35
Remark

• With 3 events, potentially there should be 23
  states
• (E,S,D)(0,0,0),(1,0,0),(0,1,0), ,(1,1,1)
• Some doesnt make sense
• E.g., ESD 111
• Name and create a state for the ones that we want
  to consider
• Wander (ESD000)
• Chase (ESD010)
• Attack (ESD1x0), x for dont-care
• Die (ESDxx1)

36
FSM (first-cut)
Problem Cant go directly from attack to chase.
Why not?

• Events
• E see an enemy
• S hear a sound
• D die
• States
• E enemy in sight
• S sound audible
• D dead

37
FSM (first-cut)
Extra state needs to be defined
AttackS 110
S
S
E

• Events
• E see an enemy
• S hear a sound
• D die
• States
• E enemy in sight
• S sound audible
• D dead

E
D
38
Quake Bot Example (refined)

• Types of behavior to capture
• Wander randomly if dont see or hear an enemy
• When see enemy, attack
• When not see enemy and hear an enemy, chase enemy
• When die, respawn
• Extensions
• When health is low and see an enemy, retreat
• When see power-ups during wandering, collect them
  hierarchical FSM

39
Example FSM with Retreat
Attack-ES E,-D,S,-L
Retreat-S -E,-D,S,L
Attack-E E,-D,-S,-L
S

• States
• E enemy in sight
• S sound audible
• D dead
• L Low health
• A lot more states got added

L
-S
L
E
-L
-E
E
Retreat-ES E,-D,S,L
-L
E
Wander-L -E,-D,-S,L
-L
E
L
-S
-L
S
L
Retreat-E E,-D,-S,L
Wander -E,-D,-S,-L
-E
-E
E
D
D
Chase -E,-D,S,-L
D
D
Spawn D (-E,-S,-L)
S
40
Hierarchical FSMs

• What if there is no simple action for a state?
• Expand a state into its own FSM, which explains
  what to do if in that state
• Some events move you around the same level in the
  hierarchy, some move you up a level
• When entering a state, have to choose a state for
  its child in the hierarchy
• Set a default, and always go to that
• Or, random choice
• Depends on the nature of the behavior

41
Hierarchical FSM Example
Attack
Wander
E
E
Chase
Pick-up Powerup
S
S
Spawn
Start
Turn Right
D
E
Go-through Door

• Note This is not a complete FSM
• All links between top level states still exist
• Need more states for wander

42
Non-Deterministic HierarchicalFSM (Markov Model)

• Adds variety to actions
• Have multiple transitions for the same event
• Label each with a probability that it will be
  taken
• Randomly choose a transition at run-time
• Markov Model New state only depends on the
  previous state

Attack
Start
43
FSM Control System Implementation
44
FSM Implementation
45
Efficient Implementation

• Compile into an array of state-name, event
• state-namei1 arraystate-namei, event
• Switch on state-name to call execution logic
• Markov Have array of possible transitions for
  every (state-name,event) pair, and choose one at
  random
• Hierarchical
• Create array for every FSM
• Have stack of states
• Classify events according to stack
• Update state which is sensitive to current event

46
FSM Advantages

• Very fast one array access
• Expressive enough for simple behaviors or
  characters that are intended to be dumb
• Can be compiled into compact data structure
• Dynamic memory current state
• Static memory state diagram array
  implementation
• Can create tools so non-programmer can build
  behavior
• Non-deterministic FSM can make behavior
  unpredictable

47
FSM Disadvantages

• Number of states can grow very fast
• Exponentially with number of events s2e
• Number of arcs can grow even faster as2
• Propositional representation
• Difficult to put in pick up the better powerup,
  attack the closest enemy
• Expensive to count Wait until the third time I
  see enemy, then attack
• Need extra events First time seen, second time
  seen, and extra states to take care of counting

48
Example
49
Code 1
Ad hoc implementation
50
Code 1p
51
Code 2
Structure, Readable, maintainable
52
Hierarchical
53
FSM Extensions
54
Stack-based FSM
Pushdown automaton (PDA)
History stack Remember previous state create
characters with a memory
55
Goal-based vs. State-based

• There is also a slight derivative to the
  state-based engine, but it used in more
  complicated games like flight simulators and
  games like MechWarrior. They use goal -based
  engines - each entity within the game is assigned
  a certain goal, be it 'protect base', 'attack
  bridge', 'fly in circles'. As the game world
  changes, so do the goals of the various entities.

56
Processing Models

• Polling
• FSM update frequency
• Easy to implement and debug
• Inefficiency (Little Red example)

• Event-driven
• Publish-subscribe messaging system
• Game engine sends event messages to individual
  FSMs
• An FSM subscribes only to the events that have
  the potential to change the current state
• Higher efficiency, non-trivial implementation

57
Interfacing with Game Engine

• Query, act on the game world
• Move the function calls (to the game engine) as
  DLL reduce the amount of recompilation required

58
Efficiency and Optimization

• In AI, FSM is the most efficient technology
  available
• Yet, there is always room for improvement
• Level of Detail depending on the condition
  (e.g., distance with player), use different FSM,
  or different update frequency

59
References

• Web references
• www.gamasutra.com/features/19970601/build_brains_i
  nto_games.htm
• csr.uvic.ca/mmania/machines/intro.htm
• www.erlang/se/documentation/doc-4.7.3/doc/design_p
  rinciples/fsm.html
• www.microconsultants.com/tips/fsm/fsmartcl.htm
• http//www.angelfire.com/dragon/letstry/tutorials/
  dfa/
• Game Programming Gems Sections 3.0 3.1
• Its very very detailed, but also some cute
  programming