Loading...

PPT – Decision Making AI PowerPoint presentation | free to download - id: 4b0354-Mjk0Z

The Adobe Flash plugin is needed to view this content

Decision Making AI

- John See
- 20 Dec 2010

Decision Making AI

- Ability of a game character to decide what to do
- Decision Making in Millingtons Model

Decision Making AI

- Decision Trees
- Finite State Machines (FSM)
- Rule-based Systems
- Fuzzy Logic Neural Networks
- Blackboard Architecture

Decision Trees

- Fast, easy to understand
- Simplest technique to implement, but extensions

to the basic algorithm can be sophisticated - Typically used to control characters, animation

or other in-game decision making - Can be learned, and learning is relatively fast

(compared to fuzzy logic/NN)

Decision Trees Problem Statement

- Given a set of knowledge, we need to generate a

corresponding action from a set of actions - Map input and output typically, a same action

is used for many different sets of input - Need a method to easily group lots of inputs

together under one particular action, allowing

the input values that are significant to control

the output

Decision Trees Problem Statement

- Example Grouping a set of inputs under an action

Enemy is visible Enemy is now lt 10m away

Attack

Enemy is visible Enemy is still far (gt 10m), but

not at flank

Attack

Enemy is visible Enemy is still far (gt 10m), at

flank

Move

Enemy is not visible, but audible

Creep

Decision Trees Algorithm Overview

- Made up of connected decision points
- Tree has starting decision, its root
- For each decision, starting from the root, one of

a set of ongoing options is chosen. - Choice is made based on characters knowledge

(internal/external) ? Fast! No prior

representation! - Continues along the tree, making choices at each

decision node until no more decisions to consider - At each leaf of the tree, an action is attached
- Action is carried out immediately

Decision Trees

- Check a single value and dont contain any

Boolean logic (AND, OR) - Representative set
- Boolean Value is true
- Enumeration Matches one of the given set of

values - Numeric value Value is within given range
- 3D Vector Vector has a length within given

range - Examples?

Decision Trees Combining Decisions

- AND two decisions place in series in the tree
- OR two decisions also use decisions in series,

but the two actions are swapped over from the AND

Decision Complexity

- Number of decisions that need to be considered is

usually much smaller than number of decisions in

the tree. - Imagine using IF-ELSE statements to test each

decision? - Method of building DTs Start with simple tree,

as AI is tested in game, additional decisions can

be added to trap special cases or add new

behaviors

Decision Making - Branching

- So far, we have considered only binary trees

decisions choose between 2 options. - It is possible to build DT with any number of

options, or different decisions with different

number of branches

Decision Making - Branching

- Deep DTs may result in a same alert being checked

numerous times before a decision is found - Flat DTs are more efficient, requires less

decision checking

Decision Making - Branching

- Still common to find DTs using only binary

decisions - Why?
- Underlying code for multiple branches simplifies

down to a series of binary tests (IF-ELSE

statements) - Binary DTs are easier to optimize. Some learning

algorithms that work with DTs require them to be

binary

Decision Making Performance

- Takes no memory, performance is linear with

number of nodes visited - Assume each decision takes constant amount of

time, and tree is balanced, performance O(log2

n), where n is number of decision nodes in tree - This DOES NOT consider the execution time of the

different checks required in the DT, which can

vary a lot!

Decision Making Balancing the Tree

- DTs can run the fastest when a tree is balanced
- A balanced tree keeps the height of its branches

approximately equal (within 1 to be considered

balanced). In our context, it will have about the

same number of decision making levels on each

branch

Decision Making Balancing the Tree

- 8 behaviors, 7 decisions
- 1st tree extremely unbalanced, 2nd tree

balanced - To get to H, 1st tree needs 8 decisions, 2nd tree

needs 3 only

Decision Making Balancing the Tree

- If all behaviors are equally likely, what is the

average number of decisions needed for both trees?

Decision Making Balancing the Tree

- If we were likely to end up at decision A

majority of time, which is more efficient? - How do we treat decisions that are time-consuming

to run? Lets say A is the most time-consuming

decision

Decision Making Merging Patterns

- DTs can be extended to allow multiple branches to

merge into a new decision efficient, but some

care is needed! - A decision/action can be reached in more than one

way - Avoid causing loops, can never find an action

leaf

Random Decision Trees

- To provide some unpredictability and variation to

making decisions in DTs - Simplest way Generate a random number and choose

a branch based on its value - DTs are normally intended to run frequently,

reacting to the game state, random decisions can

be a problem - What is a potential problem with the following

DT?

Random Decision Trees

- Possible considerations
- Allow random decision to keep track of what it

did last time. - When a decision is considered, a choice is made

at random, and that choice is stored. Next time

the decision is considered, no more randomness,

previous choice is maintained. - If something in the world changes, a different

decision was arrived, the stored choice should be

removed. - Timing Out Allow the AI to time out after a

set time, and a random choice is to be made

again. Gives variety and realism.

State Machines

- Often, characters in a game act in one of a

limited set of ways - Carry on doing the same thing until some event or

influence makes them change - Can use decision trees, but it is easier to model

this behavior using state machines (or finite

state machines, FSM) - State machines take into consideration
- the world around them
- their internal state

State Machines Basics

- Each AI character occupies one state at each

instance - Actions or behaviors are associated with each

state - So long as the character remains in that state,

it will continue carrying out the same

actions/behavior - States are connected by transitions
- Each transitions leads from one state to another,

the target state, and each has a set of

associated conditions - Changing states when the game determines that

conditions of a transition are met, the

conditions trigger and a new state is fired

State Machines Simple Example

- State machine to model a soldier 3 states
- Each state has its own transitions
- The solid circle (with a transition w/o trigger

condition) points to the initial state that will

be entered when the state machine is first run

State Machines vs. Decision Trees

- Now, name some obvious differences in making

decisions using decision trees and state

machines?

Finite State Machines (FSM)

- In game AI, a state machine with this kind of

structure (as seen earlier) is usually called a

finite state machine (FSM) - An FSM has a finite number of states and

transitions - It has finite internal memory to store its states

and transitions

FSM Generic Implementation

- Use a generic state interface that keeps track of

a set of possible states and records the current

state it is in - With each state, a series of transitions are

maintained. Each transition is also a generic

interface with conditions - At each iteration (game loop), an update function

is called to check if any of the transitions from

the current state is triggered. - If a transition is triggered, then the transition

will be fired - The separation of triggering and firing of

transitions allows the transitions to have their

own actions

FSM Generic Implementation

- Refer to textbook or other references for a more

in-depth code-level implementation of FSMs

FSM Complexity

- State machines only requires memory to hold a

triggered transition and the current state - O(1) in memory and O(m) in time, where m is the

(average) number of transitions per state - The algorithm calls other supporting functions to

perform action and etc. These probably account

for most of the time spent in the algorithm. - Hard-coded FSM inflexible, does not allow level

designers the control over building the FSM logic

Hard-coded FSM

- Hard-coded FSM
- Consists of an enumerated value, indicating which

state is currently occupied, and a function that

checks if a transition is followed - States are HARD-CODED, and limited to what was

HARD-CODED - Pros Easy and quick implementation, useful for

small FSMs - Cons
- Inflexible, does not allow level designers the

control over building the FSM logic - Difficult to maintain (alter) Large FSMs, messy

code - Every character needs to be coded its own AI

behaviors

Hierarchical State Machines

- One state machine is a powerful tool, but will

still face difficulty expressing some behaviors - Also if you wish to model somewhat different

behaviors from more than one state machines for a

single AI character - Example Modeling alarm behaviors with

hierarchical s/m (using a basic cleaning robot

state machine)

Hierarchical State Machines

- If the robot needs to get power if it runs out of

power resume its original duties after

recharging, these transition behaviors must be

added to ALL existing states to ensure robot acts

correctly

Hierarchical State Machines

- This is not exactly very efficient. Imagine if

you had to add many more concurrent behaviors

into your primary state machine?

Hierarchical State Machines

- A hierarchical state machine for the cleaning

robot - Nested states could be in more than one state

at a time - States are arranged in a hierarchy ? next state

machine down is only considered when the higher

level state machine is not responding to its alarm

Hierarchical State Machines

- H - history state node
- When the composite state (lower hierarchy) is

first entered, the H node indicates which

sub-state should be entered - If composite state already entered, then previous

sub-state is restored using the H node

Hierarchical State Machines

- Hierarchical state machine with cross hierarchy

transition - Most hierarchical s/m support transitions between

levels of the hierarchy - Lets say we want the robot to go back to refuel

when it does not find any more trash to collect

Hierarchical State Machines

- Refer to textbook for more details on its

implementation - Performance
- O(n) in memory (n is number of layers in

hierarchy) - O(nt) in time, where t is number is number of

transitions per state

DT SM

- Combining decision trees and state machines
- One approach Replace transitions from a state

with a decision tree - Leaves of DT (rather than straightforward

conditions/actions) are now transitions to other

states

DT SM

- To implement state machine without decision tree

transitions - We may need to model complex conditions that

require more checking per transition - May be time-consuming as need to check all the

time

Fuzzy Logic

- Founded by Lotfi Zadeh (1965)
- the essence of fuzzy logic is that everything is

a matter of degree - Imprecision in data
- and uncertainty in solving problems
- Fuzzy logic vs. Boolean logic
- 50-80 less rules than traditional rule-based

systems, to accomplish identical tasks - Examples Air-conditioner thermostat or washing

machine

Fuzzy Logic in Games

- Example of uses
- To control movement of bots/NPCs (to smooth out

movements based on imprecise target areas) - To assess threats posed by players (to make

further strategic decisions) - To classify player and NPCs in terms of some

useful game information (such as combat or

defensive prowess)

Crisp data Fuzzy data

- Crisp data (real numbers, value)
- Fuzzy data (a predicate or description, with

degree value) - Fuzzy logic gives a predicate a degree value.

Instead of belonging to a set of being excluded

(1 or 0, Boolean logic), everything can partially

belong to a set, and some things more belong than

others

Fuzzy sets

- Fuzzy sets the numeric value is called the

degree of membership (these values are NOT

probability values!) - For each set, a degree of membership of 1 given

to something completely in the set. Degree

membership of 0 given to something completely

outside the fuzzy set - Typical to use integers in implementation instead

of floating-point values (between 0 and 1), for

fast computation in game - Note Anything can be a member of multiple sets

at the same time

Fuzzy Control / Inference Process

- 3 basic steps in a fuzzy control or fuzzy

inference process

Step 1 - Fuzzification

- Mapping process converts crisp data (real

numbers) to fuzzy data (degree of membership) - E.g. Given a persons weight, find the degree to

which a person is underweight, overweight or at

ideal weight

Membership Functions

- Membership functions map input variables to a

degree of membership, in a fuzzy set between 0

and 1 - Any function can be used, and the shape usually

is governed by desired accuracy, the nature of

problem, or ease of implementation. - Boolean logic m/f

Membership Functions

- Grade m/f
- Reverse grade m/f
- Triangular m/f
- Trapezoid m/f

Membership Functions

- Earlier example of using a set of membership

functions to represent a persons weight

Step 2 Fuzzy rule base

- Once all inputs are expressed in fuzzy set

membership, combine them using logical fuzzy

rules to determine degree to which each rule is

true - E.g.
- Given a persons weight and activity level as

input, define rules to make a health decision - If overweight AND NOT active then frequent

exercise - If overweight AND active then moderate diet
- But having a fuzzy output such as frequency

exercise is not enough need to quantify the

amount of exercise (e.g. 3 hours per week)

Fuzzy rules

- Usually uses IF-THEN style rules
- If A then B
- A ? antecedent / premise
- B ? consequent / conclusion
- To apply usual logical operators to fuzzy input,

we need the following fuzzy axioms - A OR B MAX(A, B)
- A AND B MIN(A, B)
- NOT A 1 A

Fuzzy rules

- Earlier example on weight (and now, including

height) - Note that these fuzzy axioms (AND, OR, NOT) are

not the only definition of the logical operators.

There are other definitions that can be used

overweight AND tall MIN(0.7, 0.3)

0.3 overweight OR tall MAX(0.7, 0.3) 0.7 NOT

overweight 1 0.7 0.3 NOT tall 1 0.3

0.7 NOT (overweight AND tall) 1 MIN(0.7, 0.3)

0.7

Complete Rule Base

- With the above m/f for each input variable,

common requirement is to construct a complete set

of all possible combination of inputs. In this

case, we need 18 rules (2x3x3)

Rule evaluation (Creature example)

- We have an AI fuzzy decision making system, which

needs to evaluate whether a creature should

attack the player. Input variables range,

health, opponent ranking

Rule evaluation (Creature example)

- Rule base
- If (in melee range AND uninjured) AND NOT hard

then attack - If (NOT in melee range) AND uninjured then do

nothing - If (NOT out of range AND NOT uninjured) AND (NOT

wimp) then flee - Given specific degrees for the input variables,

we might get outputs that are like - Attack degree 0.2
- Do nothing degree 0.4
- Flee degree 0.7

Rule evaluation (Creature example)

- So what do we do with those fuzzy membership

output values?? - Missing link We also need to represent the

output variable as a fuzzy membership set!

Step 3 Defuzzification

- Defuzzification process Fuzzy output ? Crisp

output - From previous step, each rule in rule base

results in a degree of membership in some output

fuzzy set - With the numerical output we got earlier (0.2 for

attack, 0.4 for do nothing, 0.7 for flee), we

shall construct a composite output membership

function

Step 3 Defuzzification

- Not possible to be exact/accurate, but there are

methods that solve the problem as near as

possible - Using Highest Membership Function
- Choose fuzzy set which has highest degree of

membership and choose the output value that

represents each set. - 4 common points min, max, average of min/max,

bisector - Very simple to implement but coarse

defuzzification

Step 3 Defuzzification

- Blending Based on Membership
- Blend each characteristic point based on its

corresponding degree of membership - E.g. Character with 0.2 attack, 0.4 do nothing,

0.7 flee will produce crisp output given by (0.2

attack direction) (0.4 do nothing

direction) (0.7 flee direction) - Make sure that the eventual result is normalized

(otherwise result may be over-the-bounds or

unrealistic) - Common normalization technique Divide total

blended sum by the sum of fuzzy output values - Minimum values blended (Smallest of Maximum, SoM)
- Maximum values blended (Largest of Maximum, LoM)
- Average values blended (Mean of Maximum, MoM)

Step 3 Defuzzification

- Center of Gravity
- Also known as Centroid of Area method ? Takes

into account all membership values, rather than

specific ones (largest, smallest, average, etc.) - First, each m/f is cropped at the membership

value of its set - Center of mass is found by integrating each in

turn. This point is chosen as output crisp value - Unlike bisector of area method, we cant compute

this offline since we do not know in advance the

fuzzy membership values, and how the m/f will be

cropped

Misc Dealing with Complex Rule Base

- We may have multiple rules in our rule base that

will results in the same output membership fuzzy

set. - E.g.
- Corner-entry AND going-slow THEN accelerate
- Corner-exit AND going-fast THEN accelerate
- Corner-exit AND going-slow THEN accelerate
- How do we deal with such situations? Which output

membership value for accelerate to choose?

Some Good Examples

- 2 Examples Control Example Threat Assessment

Example from AI for Game Developers ref book