From the Ground Up: A.I. Architecture and Design Patterns

1
From the Ground Up A.I. Architecture and Design
Patterns

• Dr Brett Laming
• Rockstar Leeds

2
Contents

• Basic Principles
• Think/Act - The I/O Divide
• An Algorithmic Approach
• A Functional Approach
• Movement
• Animation
• Conclusion

3
Basic Principles
4
Why Do We Care?

• Its a Competitive Market
• Higher Expectations
• Tighter Schedules
• Multiple Platforms
• Simultaneous Titles

• But Patterns Are Everywhere
• Identify, generalise, reuse, evolve
• ? Reliability
• ? Production Speed
• ? More Fun Stuff

5
Inspirations
Marvin Minsky Different representations for
different views
Daniel Dennet Behaviour can be viewed at the
physical, design and intentional levels David
Marr Computational, algorithmic and
implementational
Damian Isla Cognitive Maps Spatial
Relations Semantics
Noam Chomsky Hierarchical decomposition Structure
vs Meaning Colorless green ideas sleep furiously
Chris Hecker Style vs Structure 2008 What is
the textured triangle of A.I.
Craig Reynolds Simple rules Complex behaviour
6
What Am I Looking For?
Algorithms
Concepts
Hierarchy
Components
Reusability
Best Practice
Commutability
7
How Do I Find Them?

• Observation
• How might it work?
• Introspection
• What would I do?
• Generalisation
• Ive done this before
• They do the same

• Bad Experience
• Lets not do that again
• Background
• I studied this once?
• Could I apply this?

8
Best Practice
9
Best Practice

• Prototype new ideas where possible
• Get visual and design direction
• Mock-ups
• Prove (or disprove) the concept
• Quick and dirty programming
• Play to peoples strengths
• Maths ? Physics Guys
• Navigation Mesh ? Collision Guys

10
Best Practice

• Program Defensively
• Assert and Unit Test
• Automated Scripts as soak tests
• One co-ordinate system and S.I. units
• Maximise Workflow
• Cater tools to their needs
• Put new functionality on bypass
• Think of the man-hour cost!
• Minimise potential for human error

11
Best Practice

• Build A Debugging Suite
• Instant Pause
• Flyable Camera
• Layered Information
• Action Histories
• Maintain Player Immersion
• A.I. should not be too bad or too good
• Limit ourselves to what the player would know
• Constrain them to the same actions

12
The Algorithmic Approach
13
The Think-Act Loop
Player
A.I.

• Key tenants
• Use only what we might know
• Mimic the player

Sensory Receptors
Brain
Controller
Game
14
Sensory Input
Sensory Data
Think
Controller
Act
15
Sensory Data

• Detail level
• Depends on genre and perceived communication
• Thief vs Medieval Total War II
• General Model
• Visual component an arc
• Auditory component a radius
• Auditory targets less official
• Occlusion too expensive?
• Shoot a weapon to get info
• Theorise using ghost images

16
Blackboards

• Used To Share Information
• Static blackboard stores defined types of info
• Dynamic blackboard stores arbitrary data
• Agents write to the board
• Generally read it as well

Agent A
Agent B
10,20,15
-30,20,13
Scout 3, 17, 10 Cover 17, 11,5
-10,15,12
17,11,5
17
Control Output
cSensorCone
Think
Controller
Act
18
Virtual Controller (yoke)

• Purpose
• Carries control instructions
• Provides a strict I/O Divide
• Notice the const correctness
• Unifies player and A.I.
• Control mapping for the player
• A.I. fills it in from Think()
• Key Properties
• Never stored in its entirety
• Created on the stack
• Lifetime of a single Process()

Process()
cVirtualYoke yoke
Think(yoke) const
Controller
Act(const yoke)
19
So Actually
20
Virtual Yoke

• Composite
• Collection of smaller yokes
• Allows selective storage

cVirtualYoke cLocomotionYoke mLocomotion cWeapon
Yoke mWeapons cAnimYoke mAnimation
21
Taking Action
Process()
cBlackboard
cVirtualYoke yoke
Think(yoke) const
cSensorCone
Act(const yoke)
22
Act

• Applies Yoke Commands
• Composite yokes ? Subsystems
• Object Models
• Supply common interfaces
• Could be a turret mounted weapon or my pistol.
• Could be driving a car, a plane, a boat or
  myself!

23
Object Model

• Self Contained
• Instructions for Think
• Actions for Act
• Commutable
• Plug and Play
• Downloadable Content
• Maybe broadcast use
• Think Sims 2!

iLocomotive // Fills in yoke ComputeMotion(const
cTarget , cLocomotiveYoke) // Computes
forces ApplyMotion(const cTarget ,
const cLocomotiveYoke, v3 force,
v3 torque)
24
More Coolness
ApplyMotion(const cTarget targ,
cLocomotiveYoke yoke) v3 force(Zero) v3
torque(Zero) if (Drivable())
Drivable()-gtApplyMotion( targ, yoke, force,
torque) if (Floatable()) Floatable()-gtAppl
yMotion( targ, yoke, force, torque)
ApplyForceTorque(force, torque)
Compute
Apply
25
Frame Time
Process()
cBlackboard
cVirtualYoke yoke
dt()
Think(yoke, dt) const
cSensorCone Update(dt)
Act(const yoke, dt)
26
dt()

• Work into everything
• Including fixed time steps
• No more sv, svdt()
• Benefits
• Integration
• Implicit forward Euler
• Rough Smoothing
• Closed Feedback
• Pause dt0
• Level Of Detail dt2dt
• Special Effects

smoothing
feedback control
dt()
27
The Functional Approach
28
Problem Domain
Needs a problem ?

• Examine the Terminology
• Feeling, Knowledge, Goals, Beliefs, Needs
• Examine the Concepts
• Decisions, Facts, Uncertainty, Exploration,
  Verb-Noun Actions, Repetition, Sequencing

29
What Are We Doing?

• Goal based reasoning
• Working to solve a goal
• Thinking about and realising smaller tasks
• Taking a hierarchical approach
• Using a limited number of short verb-noun
  pairings to form a plan
• Weve seen this before
• An old pattern
• Colossal Cave Adventure and MUDs
• Verb-noun actions like get axe, wield axe
  separated by movement n, e, e, s, e
• We use a container object model

30
Applying The Pattern
Taking the computational to the algorithmic

• Know of Mine, Smithy
• Goto Mine
• Know of Wall
• Get Ore From Wall
• Goto Smithy
• Know of Door
• Use Door
• Play Open Door
• Warp Inside
• Know of Owner, Forge
• Play Close Door

• Goto Owner
• Put 10 gp In Owner
• Get Time From Owner
• Goto Forge
• Put Time In Forge
• Put Iron In Forge
• Use Forge
• Use Enchant
• Use Forge
• Get Sword From Anvil

31
Easy Questions

• Why did I choose to do this again?
• Because we were driven to it by personality and
  need
• What happens when I get the treasure?
• Ill probably choose to do something else
  depending on my mood
• Why stop at get iron?
• Because its reached the atomic level - there are
  no more questions, just results

32
Putting It Together
Ambient Controller Get treasure
Planning Get magic sword
33
Explicit Orders

• Script commands
• Script on A.I.
• Autopilot
• Player on Player
• Player instructions
• Player on A.I.
• Squad Commander
• A.I. on A.I.

Ambient Controller Get treasure
Planning Get magic sword
34
Scripting Notes

• Dont mix styles
• Script has immediate control
• Script waits for an opportunity
• Keep common properties separate
• No sharing memory locations
• A script global population density
• A code global population density
• Maintain a set order of calculation
• Generally consistent with style

35
Ambient Controller

• Generates sensible actions autonomously
• Maybe Idle
• Maybe Full Daily Routine
• Daily Routines
• Character properties
• Needs/Drives
• Scheduling
• Time of day
• nice emergent behaviour

Ambient Controller Get treasure
Planning Get magic sword
36
Daily Routine
Sleep
Time Of Day
Drive Model
Goto Work
Work
Character
Leave Work
Relax
Schedule
Go Home
37
Planning
Ambient Controller Get treasure
Planning Get magic sword

• Continuous bar dt()
• Defines Think()

38
Plan Components

• Play Animation
• Waits on dt()
• Use Object
• Object model again
• Broadcast actions.
• Change world based on state
• May wait on dt()

• Get/Take Object From Container
• Primary world manipulation
• Contents determine state
• Goto Location
• Waits on dt()
• Key A.I. output
• Complex
• Warrants special attention later

39
Search Based Planning

• Traditional academic approach
• See STRIPS, Hierarchical Task Networks, Bratko
• The Good
• Mimics our introspective reasoning
• Seeks to fully realise a plan to the goal
• Directed search for optimal solutions
• Post processing even more so
• The Bad
• Knowledge representation
• Scalability - difficult for video games
• Lots of storage

40
Procedural Planning

• Industry preferred approach
• Hierarchical
• Easy to comprehend
• Limited Language
• Goals
• Sub-goals
• Conditions
• Actions
• Transitions
• Sequential
• Decision Based
• Powerful Results

Get Object
Short of money? Make
Thief skills? Steal
Buy
Get Ore
Goto Mine
41
Procedural Planning - Issues

• Competing children
• We have to make a best guess from the options
• A might help
• But we could still end up with case of a basic
  sword being bought but not affording the forge.
• Incomplete plan means no post process
• Not good for player supporting A.I.
• More action for generic A.I.

42
The Curve Ball

• Task Interruption
• Im returning to my gang hideout
• I see an enemy
• I engage the enemy
• I roll out of the way of a car
• I recover to my feet
• I re-engage the enemy
• I continue to return to my gang hideout
• Is A Key Requirement of our A.I.

43
Finite State Machines

• No plan history
• No idea of context
• No generalised exit.
• Hideous state history workarounds
• Dont scale well
• Many transitions

44
Behaviour Trees
Get Object
Decorators
Selection
Crime Allowed?
Create
Buy
Low money?
Get Material
Steal
Ok money?
Sequencing
Thief Class?
Goto Source
Take Material
Preconditions
Actions
45
Modelling Interruption?
Ambient Behaviour
Kill Enemy
Return Home
Parallel?
Goto Location
Locate Hideout
Gain Range
Fire Weapon
46
Behaviour Trees

• Simple and powerful
• Limited vocabulary
• Most situations handled
• Highly flexible
• Plug and play
• Customisable
• Nice design tools
• Handy child evaluation
• Lends itself to directed decision making

• Issues
• Interruption handling
• Where to return
• Amount of flexibility
• Trees get complicated

47
MARPO
Get Object
Thief class? Thieving allowed?
Not enough money?
Money?
Steal
Create
Buy
Get Material
Goto Source
Take Material
48
MARPO
Reactive
Long term
Immediate
Get Object
Create
Get Material
Take Material
49
MARPO
Reactive
Long term
Immediate
Get Object
Kill Enemy
50
MARPO
Reactive
Long term
Immediate
Get Object
Kill Enemy
Roll
51
MARPO

• Behavioural Bamboo Forest
• Stack based tasks
• Suppression model
• Multiple threads of execution
• Keeps only one stack in memory down to current
  task
• Decision logic lies with the parent
• Higher level parameterised building blocks
• Authoring is by script not tool.
• Winding Ability
• Allows auto-recovery from any state
• Respects script orders on an immediate basis

52
Movement
53
Goto Position

• What Is Position?
• A world co-ordinate
• An entity
• A navigation point
• An offset off an entity
• A radius off an offset, off an entity
• Still World Positions!
• So generalise

cTarget void Set() void SetArrivalConditions()
v3 WorldPos() const v3 InterceptPos()
const bool HasArrived() const

• Lots for free
• Entity intercept
• Completion checks

54
The Problem
55
Stage 1 - Navigation

• Navigation Mesh
• Industry standard Tozour, 2008
• Handles static geometry
• Searching with A
• Optimise search space, not A
• Improve Data Format
• Make it Hierarchical
• Allow for reference spaces Isla, 2005

56
Navigation

• Quick To Search
• Live Updates
• Zones
• Spatial Reasoning Isla, 2005

57
Components
cNavPos
cNavPoly indices edge info cNavId
mParent cNavId mFirstChild mChildCount
cNavigator Pos() FinalTarget() CurrentTarget() Ha
sArrived() Update( cTarget ) cNavPos
mFinal cNavPos mCurrent cNavPos mPos
cNavId mPolyId mPolySource
cPolyCoords local co-ords
58
Stage 2 - Optimisation

• String Pulling
• Bevel uncrossable edges to a radius
• Tighten points

59
Stage 3 - Dynamic Avoidance

• At rest
• Mesh binding
• On move
• Intercept calculation

60
Stage 4 - Locomotion

• Differing approach
• Vehicle
• Humanoid
• Can we generalise?

61
Locomotion

• Generalised by iLocomotion
• Same interface
• Different yoke instructions
• Vehicles
• Vehicles supply gas, steering
• Difficulty is in mapping target to gas and
  steering
• Actors
• Actors supply ideal position, velocity, direction
• Difficulty is in animation to hit position.

iLocomotion ComputeMotion() ApplyMotion()
62
Locomotion - Vehicles
Angle To Steering Maths PID Controller
Target Position
Yoke
Speed For Deceleration
Speed For Steering
Top Speed
Min
Speed to Gas Maths PID Controller
Yoke
63
Some Hints ?
Speed For Deceleration
Speed to Gas Angle To Steering
Manual Shift

• Equations of Motion

• PID

• Response Curves

64
Animation
65
Locomotion - Character

• Animation ? Position
• Challenges
• Sensible manoeuvre choice
• Animation Graphs
• Navigation Mark Up
• Natural fluidity
• Foot positioning
• Hand positioning

Idle
Crouch
Walk
Crouch Walk
Run
Crouch Run
66
Goal Based

• Directed Search of Animation Graph
• Incorporates interesting A.I. along the way
• Think Gears of War, A
• Interested ? Champandard
• Personal Issues
• Need to find a full animation plan to goal
• This would need to sit with path finding
• Makes it more expensive
• Painful Heuristic Balancing
• Switching to the time domain probably partially
  solves it
• But how do we prevent repeated actions

67
Carrot On A Stick Method

• Keep iLocomotion interface
• ComputeMotion() computes velocity
• ApplyMotion() animates for velocity
• dt() helps minimise error
• Want to use a cover point?
• Explicit decision
• a new target

iLocomotion ComputeMotion() ApplyMotion()

• Given the animation properties
• We can solve by mathematics

68
Carrot On A Stick Method
Kill
Under fire?
Cover
Find cover?
Nav. mesh
Route to cover
Anchor Point
69
Carrot On A Stick Method
Direction for Animation
70
Carrot On A Stick Method
s
71
Carrot On A Stick Method

• Animations
• distance (s) in time (t)
• an average velocity
• Assume velocity bands
• Look up v for animation
• Dead zones indirectly determine blending
• a and d now character properties
• Actively correct for v
• Use tricks like light I.K.
• Implicitly correct by dt()

v
Crouch run
Crouch walk
Crouch run
v
Crouch walk
t
72
Carrot On A Stick Method

• Animation graph
• Used for information query only
• Locomotive cycle driven by vaim
• Free things!
• Foot positioning
• Obtained through cycle alteration and dt()
• Blending
• Character movement properties
• Obtained through a and d
• Swim, crawl, crouch, walk
• Same procedure, different velocity bands
• Human like arrival mistakes

73
Conclusion
74
Conclusion
Good reusable AI is about

• Best Practices
• Strict interfaces
• Multiple takes on problems
• Concepts
• Commutability

• Components
• Algorithms
• Hierarchy
• Pattern Recognition
• Human Sciences

75
Conclusion

• Style vs Structure - Hecker 2008
• Texture Mapped A.I. Triangle
• Style
• No idea where it is annoyingly
• Structure
• Not one single triangle
• But many triangle like pieces
• Represented differently
• But all essentially the same piece