Decision Making AI - PowerPoint PPT Presentation


PPT – Decision Making AI PowerPoint presentation | free to download - id: 4b0354-Mjk0Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Decision Making AI


In this case, we need 18 rules (2x3x3) Rule evaluation (Creature example) We have an AI fuzzy decision making system, which ... Decision Making ... – PowerPoint PPT presentation

Number of Views:61
Avg rating:3.0/5.0
Slides: 62
Provided by: pesonaMm1
Learn more at:
Tags: decision | fuzzy | making


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Decision Making AI

Decision Making AI
  • John See
  • 20 Dec 2010

Decision Making AI
  • Ability of a game character to decide what to do
  • Decision Making in Millingtons Model

Decision Making AI
  • Decision Trees
  • Finite State Machines (FSM)
  • Rule-based Systems
  • Fuzzy Logic Neural Networks
  • Blackboard Architecture

Decision Trees
  • Fast, easy to understand
  • Simplest technique to implement, but extensions
    to the basic algorithm can be sophisticated
  • Typically used to control characters, animation
    or other in-game decision making
  • Can be learned, and learning is relatively fast
    (compared to fuzzy logic/NN)

Decision Trees Problem Statement
  • Given a set of knowledge, we need to generate a
    corresponding action from a set of actions
  • Map input and output typically, a same action
    is used for many different sets of input
  • Need a method to easily group lots of inputs
    together under one particular action, allowing
    the input values that are significant to control
    the output

Decision Trees Problem Statement
  • Example Grouping a set of inputs under an action

Enemy is visible Enemy is now lt 10m away
Enemy is visible Enemy is still far (gt 10m), but
not at flank
Enemy is visible Enemy is still far (gt 10m), at
Enemy is not visible, but audible
Decision Trees Algorithm Overview
  • Made up of connected decision points
  • Tree has starting decision, its root
  • For each decision, starting from the root, one of
    a set of ongoing options is chosen.
  • Choice is made based on characters knowledge
    (internal/external) ? Fast! No prior
  • Continues along the tree, making choices at each
    decision node until no more decisions to consider
  • At each leaf of the tree, an action is attached
  • Action is carried out immediately

Decision Trees
  • Check a single value and dont contain any
    Boolean logic (AND, OR)
  • Representative set
  • Boolean Value is true
  • Enumeration Matches one of the given set of
  • Numeric value Value is within given range
  • 3D Vector Vector has a length within given
  • Examples?

Decision Trees Combining Decisions
  • AND two decisions place in series in the tree
  • OR two decisions also use decisions in series,
    but the two actions are swapped over from the AND

Decision Complexity
  • Number of decisions that need to be considered is
    usually much smaller than number of decisions in
    the tree.
  • Imagine using IF-ELSE statements to test each
  • Method of building DTs Start with simple tree,
    as AI is tested in game, additional decisions can
    be added to trap special cases or add new

Decision Making - Branching
  • So far, we have considered only binary trees
    decisions choose between 2 options.
  • It is possible to build DT with any number of
    options, or different decisions with different
    number of branches

Decision Making - Branching
  • Deep DTs may result in a same alert being checked
    numerous times before a decision is found
  • Flat DTs are more efficient, requires less
    decision checking

Decision Making - Branching
  • Still common to find DTs using only binary
  • Why?
  • Underlying code for multiple branches simplifies
    down to a series of binary tests (IF-ELSE
  • Binary DTs are easier to optimize. Some learning
    algorithms that work with DTs require them to be

Decision Making Performance
  • Takes no memory, performance is linear with
    number of nodes visited
  • Assume each decision takes constant amount of
    time, and tree is balanced, performance O(log2
    n), where n is number of decision nodes in tree
  • This DOES NOT consider the execution time of the
    different checks required in the DT, which can
    vary a lot!

Decision Making Balancing the Tree
  • DTs can run the fastest when a tree is balanced
  • A balanced tree keeps the height of its branches
    approximately equal (within 1 to be considered
    balanced). In our context, it will have about the
    same number of decision making levels on each

Decision Making Balancing the Tree
  • 8 behaviors, 7 decisions
  • 1st tree extremely unbalanced, 2nd tree
  • To get to H, 1st tree needs 8 decisions, 2nd tree
    needs 3 only

Decision Making Balancing the Tree
  • If all behaviors are equally likely, what is the
    average number of decisions needed for both trees?

Decision Making Balancing the Tree
  • If we were likely to end up at decision A
    majority of time, which is more efficient?
  • How do we treat decisions that are time-consuming
    to run? Lets say A is the most time-consuming

Decision Making Merging Patterns
  • DTs can be extended to allow multiple branches to
    merge into a new decision efficient, but some
    care is needed!
  • A decision/action can be reached in more than one
  • Avoid causing loops, can never find an action

Random Decision Trees
  • To provide some unpredictability and variation to
    making decisions in DTs
  • Simplest way Generate a random number and choose
    a branch based on its value
  • DTs are normally intended to run frequently,
    reacting to the game state, random decisions can
    be a problem
  • What is a potential problem with the following

Random Decision Trees
  • Possible considerations
  • Allow random decision to keep track of what it
    did last time.
  • When a decision is considered, a choice is made
    at random, and that choice is stored. Next time
    the decision is considered, no more randomness,
    previous choice is maintained.
  • If something in the world changes, a different
    decision was arrived, the stored choice should be
  • Timing Out Allow the AI to time out after a
    set time, and a random choice is to be made
    again. Gives variety and realism.

State Machines
  • Often, characters in a game act in one of a
    limited set of ways
  • Carry on doing the same thing until some event or
    influence makes them change
  • Can use decision trees, but it is easier to model
    this behavior using state machines (or finite
    state machines, FSM)
  • State machines take into consideration
  • the world around them
  • their internal state

State Machines Basics
  • Each AI character occupies one state at each
  • Actions or behaviors are associated with each
  • So long as the character remains in that state,
    it will continue carrying out the same
  • States are connected by transitions
  • Each transitions leads from one state to another,
    the target state, and each has a set of
    associated conditions
  • Changing states when the game determines that
    conditions of a transition are met, the
    conditions trigger and a new state is fired

State Machines Simple Example
  • State machine to model a soldier 3 states
  • Each state has its own transitions
  • The solid circle (with a transition w/o trigger
    condition) points to the initial state that will
    be entered when the state machine is first run

State Machines vs. Decision Trees
  • Now, name some obvious differences in making
    decisions using decision trees and state

Finite State Machines (FSM)
  • In game AI, a state machine with this kind of
    structure (as seen earlier) is usually called a
    finite state machine (FSM)
  • An FSM has a finite number of states and
  • It has finite internal memory to store its states
    and transitions

FSM Generic Implementation
  • Use a generic state interface that keeps track of
    a set of possible states and records the current
    state it is in
  • With each state, a series of transitions are
    maintained. Each transition is also a generic
    interface with conditions
  • At each iteration (game loop), an update function
    is called to check if any of the transitions from
    the current state is triggered.
  • If a transition is triggered, then the transition
    will be fired
  • The separation of triggering and firing of
    transitions allows the transitions to have their
    own actions

FSM Generic Implementation
  • Refer to textbook or other references for a more
    in-depth code-level implementation of FSMs

FSM Complexity
  • State machines only requires memory to hold a
    triggered transition and the current state
  • O(1) in memory and O(m) in time, where m is the
    (average) number of transitions per state
  • The algorithm calls other supporting functions to
    perform action and etc. These probably account
    for most of the time spent in the algorithm.
  • Hard-coded FSM inflexible, does not allow level
    designers the control over building the FSM logic

Hard-coded FSM
  • Hard-coded FSM
  • Consists of an enumerated value, indicating which
    state is currently occupied, and a function that
    checks if a transition is followed
  • States are HARD-CODED, and limited to what was
  • Pros Easy and quick implementation, useful for
    small FSMs
  • Cons
  • Inflexible, does not allow level designers the
    control over building the FSM logic
  • Difficult to maintain (alter) Large FSMs, messy
  • Every character needs to be coded its own AI

Hierarchical State Machines
  • One state machine is a powerful tool, but will
    still face difficulty expressing some behaviors
  • Also if you wish to model somewhat different
    behaviors from more than one state machines for a
    single AI character
  • Example Modeling alarm behaviors with
    hierarchical s/m (using a basic cleaning robot
    state machine)

Hierarchical State Machines
  • If the robot needs to get power if it runs out of
    power resume its original duties after
    recharging, these transition behaviors must be
    added to ALL existing states to ensure robot acts

Hierarchical State Machines
  • This is not exactly very efficient. Imagine if
    you had to add many more concurrent behaviors
    into your primary state machine?

Hierarchical State Machines
  • A hierarchical state machine for the cleaning
  • Nested states could be in more than one state
    at a time
  • States are arranged in a hierarchy ? next state
    machine down is only considered when the higher
    level state machine is not responding to its alarm

Hierarchical State Machines
  • H - history state node
  • When the composite state (lower hierarchy) is
    first entered, the H node indicates which
    sub-state should be entered
  • If composite state already entered, then previous
    sub-state is restored using the H node

Hierarchical State Machines
  • Hierarchical state machine with cross hierarchy
  • Most hierarchical s/m support transitions between
    levels of the hierarchy
  • Lets say we want the robot to go back to refuel
    when it does not find any more trash to collect

Hierarchical State Machines
  • Refer to textbook for more details on its
  • Performance
  • O(n) in memory (n is number of layers in
  • O(nt) in time, where t is number is number of
    transitions per state

  • Combining decision trees and state machines
  • One approach Replace transitions from a state
    with a decision tree
  • Leaves of DT (rather than straightforward
    conditions/actions) are now transitions to other

  • To implement state machine without decision tree
  • We may need to model complex conditions that
    require more checking per transition
  • May be time-consuming as need to check all the

Fuzzy Logic
  • Founded by Lotfi Zadeh (1965)
  • the essence of fuzzy logic is that everything is
    a matter of degree
  • Imprecision in data
  • and uncertainty in solving problems
  • Fuzzy logic vs. Boolean logic
  • 50-80 less rules than traditional rule-based
    systems, to accomplish identical tasks
  • Examples Air-conditioner thermostat or washing

Fuzzy Logic in Games
  • Example of uses
  • To control movement of bots/NPCs (to smooth out
    movements based on imprecise target areas)
  • To assess threats posed by players (to make
    further strategic decisions)
  • To classify player and NPCs in terms of some
    useful game information (such as combat or
    defensive prowess)

Crisp data Fuzzy data
  • Crisp data (real numbers, value)
  • Fuzzy data (a predicate or description, with
    degree value)
  • Fuzzy logic gives a predicate a degree value.
    Instead of belonging to a set of being excluded
    (1 or 0, Boolean logic), everything can partially
    belong to a set, and some things more belong than

Fuzzy sets
  • Fuzzy sets the numeric value is called the
    degree of membership (these values are NOT
    probability values!)
  • For each set, a degree of membership of 1 given
    to something completely in the set. Degree
    membership of 0 given to something completely
    outside the fuzzy set
  • Typical to use integers in implementation instead
    of floating-point values (between 0 and 1), for
    fast computation in game
  • Note Anything can be a member of multiple sets
    at the same time

Fuzzy Control / Inference Process
  • 3 basic steps in a fuzzy control or fuzzy
    inference process

Step 1 - Fuzzification
  • Mapping process converts crisp data (real
    numbers) to fuzzy data (degree of membership)
  • E.g. Given a persons weight, find the degree to
    which a person is underweight, overweight or at
    ideal weight

Membership Functions
  • Membership functions map input variables to a
    degree of membership, in a fuzzy set between 0
    and 1
  • Any function can be used, and the shape usually
    is governed by desired accuracy, the nature of
    problem, or ease of implementation.
  • Boolean logic m/f

Membership Functions
  • Grade m/f
  • Reverse grade m/f
  • Triangular m/f
  • Trapezoid m/f

Membership Functions
  • Earlier example of using a set of membership
    functions to represent a persons weight

Step 2 Fuzzy rule base
  • Once all inputs are expressed in fuzzy set
    membership, combine them using logical fuzzy
    rules to determine degree to which each rule is
  • E.g.
  • Given a persons weight and activity level as
    input, define rules to make a health decision
  • If overweight AND NOT active then frequent
  • If overweight AND active then moderate diet
  • But having a fuzzy output such as frequency
    exercise is not enough need to quantify the
    amount of exercise (e.g. 3 hours per week)

Fuzzy rules
  • Usually uses IF-THEN style rules
  • If A then B
  • A ? antecedent / premise
  • B ? consequent / conclusion
  • To apply usual logical operators to fuzzy input,
    we need the following fuzzy axioms
  • A OR B MAX(A, B)
  • A AND B MIN(A, B)
  • NOT A 1 A

Fuzzy rules
  • Earlier example on weight (and now, including
  • Note that these fuzzy axioms (AND, OR, NOT) are
    not the only definition of the logical operators.
    There are other definitions that can be used

overweight AND tall MIN(0.7, 0.3)
0.3 overweight OR tall MAX(0.7, 0.3) 0.7 NOT
overweight 1 0.7 0.3 NOT tall 1 0.3
0.7 NOT (overweight AND tall) 1 MIN(0.7, 0.3)
Complete Rule Base
  • With the above m/f for each input variable,
    common requirement is to construct a complete set
    of all possible combination of inputs. In this
    case, we need 18 rules (2x3x3)

Rule evaluation (Creature example)
  • We have an AI fuzzy decision making system, which
    needs to evaluate whether a creature should
    attack the player. Input variables range,
    health, opponent ranking

Rule evaluation (Creature example)
  • Rule base
  • If (in melee range AND uninjured) AND NOT hard
    then attack
  • If (NOT in melee range) AND uninjured then do
  • If (NOT out of range AND NOT uninjured) AND (NOT
    wimp) then flee
  • Given specific degrees for the input variables,
    we might get outputs that are like
  • Attack degree 0.2
  • Do nothing degree 0.4
  • Flee degree 0.7

Rule evaluation (Creature example)
  • So what do we do with those fuzzy membership
    output values??
  • Missing link We also need to represent the
    output variable as a fuzzy membership set!

Step 3 Defuzzification
  • Defuzzification process Fuzzy output ? Crisp
  • From previous step, each rule in rule base
    results in a degree of membership in some output
    fuzzy set
  • With the numerical output we got earlier (0.2 for
    attack, 0.4 for do nothing, 0.7 for flee), we
    shall construct a composite output membership

Step 3 Defuzzification
  • Not possible to be exact/accurate, but there are
    methods that solve the problem as near as
  • Using Highest Membership Function
  • Choose fuzzy set which has highest degree of
    membership and choose the output value that
    represents each set.
  • 4 common points min, max, average of min/max,
  • Very simple to implement but coarse

Step 3 Defuzzification
  • Blending Based on Membership
  • Blend each characteristic point based on its
    corresponding degree of membership
  • E.g. Character with 0.2 attack, 0.4 do nothing,
    0.7 flee will produce crisp output given by (0.2
    attack direction) (0.4 do nothing
    direction) (0.7 flee direction)
  • Make sure that the eventual result is normalized
    (otherwise result may be over-the-bounds or
  • Common normalization technique Divide total
    blended sum by the sum of fuzzy output values
  • Minimum values blended (Smallest of Maximum, SoM)
  • Maximum values blended (Largest of Maximum, LoM)
  • Average values blended (Mean of Maximum, MoM)

Step 3 Defuzzification
  • Center of Gravity
  • Also known as Centroid of Area method ? Takes
    into account all membership values, rather than
    specific ones (largest, smallest, average, etc.)
  • First, each m/f is cropped at the membership
    value of its set
  • Center of mass is found by integrating each in
    turn. This point is chosen as output crisp value
  • Unlike bisector of area method, we cant compute
    this offline since we do not know in advance the
    fuzzy membership values, and how the m/f will be

Misc Dealing with Complex Rule Base
  • We may have multiple rules in our rule base that
    will results in the same output membership fuzzy
  • E.g.
  • Corner-entry AND going-slow THEN accelerate
  • Corner-exit AND going-fast THEN accelerate
  • Corner-exit AND going-slow THEN accelerate
  • How do we deal with such situations? Which output
    membership value for accelerate to choose?

Some Good Examples
  • 2 Examples Control Example Threat Assessment
    Example from AI for Game Developers ref book