Chapter 4: Heuristic Search

1
Chapter 4 Heuristic Search

• According to Newell and Simon, Intelligence for
  a system with limited processing resources
  consists in making wise choices of what to do
  next.
• Heuristics are used to make search more
  intelligent and efficient
• A heuristic, in general, is a rule of discovery
  or invention
• A heuristic, in the practical sense, may be
  thought of as an informed guess about the next
  step a system should take to solve a problem
• Heuristics are especially useful when
• a problem does not have an exact solution
• it would take too long to find an exact solution
• Even good heuristics can fail sometimes
• If you knew how to always get the best answer
  quickly, there would be no point to using a
  heuristic in the first place

2
Heuristic Algorithms

• You can think of a heuristic algorithm as having
  two parts
• the heuristic measure (what you base the guess
  on)
• the algorithm that searches the state space,
  using that measure
• We have already seen that a good state space
  representation makes search more efficient
• For example, in tic tac toe, you could represent
  symmetric moves only once
• The heuristic algorithm makes a further
  improvement, by visiting fewer of the nodes in
  the state space
• Some heuristics are simple
• In tic tac toe, you could always move to the
  square that provides the most possible ways to
  win
• This is easy to count, by seeing how many lines
  can be drawn through the square

3
Symmetric State Space for Tic Tac Toe
4
The Most Wins Heuristic in Tic Tac Toe
5
Hill Climbing Search

• Hill climbing is a simple algorithm for heuristic
  search
• In hill climbing, you start at the root and use
  the heuristic measure on each child of the root
• Evaluating the children of a node is called
  expanding the node
• You take the best child, as measured, and
  repeat the process from there, treating the best
  child as the new root
• You discard the root and all less-good children
  from further consideration
• You stop when a node is better than any of its
  children
• This doesnt work for all problems, because you
  can get stuck in a local maximum, the top of a
  small hill, although bigger hills are nearby
• For the right search space, this can be simple
  and effective
• Worked great for finding hot spots in GE
  headlights, because laws of physics do not permit
  multiple hot spots

6
Dynamic Programming

• Dynamic programming is another relatively simple
  heuristic approach that is borrowed from
  Operations Research
• It draws its strength from reusing subproblems to
  solve larger problems that would otherwise be
  more complex in terms of time and space
• It stores the intermediate results from solving
  subproblems in an array and computes later array
  values from earlier ones
• For a simple example, consider Fibonacci numbers
• f(0) 0, f(1) 1, f(n) f(n-1) f(n-2) for n
  gt 2
• If we solve f(n) recursively, complexity becomes
  exponential as we solve the same subproblems
  repeatedly
• If we build an array initialized with f(0) and
  f(1), we can compute each new value from the two
  preceding values, with only linear complexity to
  find f(n)

7
Dynamic Programming in AI

• In AI, dynamic programming finds its greatest
  usage in natural language processing
• An example is trying to find the closest real
  word to a misspelled word in a document
• For this, we could use the minimum edit distance
  heuristic
• We assign costs to the changes we need to make to
  convert one word into another
• For example, we could count inserting or deleting
  one letter as having a cost of 1, and replacing
  one letter with another as having a cost of 2
• We add up all of the costs for the changes to
  each word, making as few changes as possible
• Then, the word or words with the smallest edit
  distance (costs) would be the most likely
  replacement(s) for the misspelled word

8
Minimum Edit Distance Example

• The minimum edit distance between the two words
  intention and execution
• can be found
• intention
• ntention delete i, cost 1
• etention replace n with e, cost
  2
• exention replace t with x, cost 2
• exenution insert u, cost 1
• execution replace n with c, cost 2
• Dynamic programming can be used to find the
  minimum edit distance using
• a two dimensional array. The value at each
  location in the array is determined
• as the minimum cost of changes up to that point,
  plus the minimum cost of
• the next insertion, deletion or replacement.

9
Best-First Search

• Best-first search shares much in common with
  depth-first, breadth-first and hill climbing
  search
• Instead of always going down or across a graph,
  as in depth-first or breadth-first search,
  best-first search always expands the most
  promising node first, according to an evaluation
  function, as in hill climbing search
• Instead of getting stuck in local maxima like
  hill climbing search, best-first search maintains
  open and closed lists, like depth-first and
  breadth-first search
• This allows it to expand the best node next,
  regardless of whether or not the best node is a
  child of the current node
• Nodes on the open list are sorted by order of
  goodness, so that the open list becomes a
  priority queue
• A node on the closed list may be reopened, if it
  turns out theres a shorter path to it than the
  first path tried

10
Best-First Search Algorithm
11
Best-First Search Example
12
Heuristic Evaluation Functions

• Some creativity is needed to find a heuristic
  evaluation function for a problem
• The function isnt usually obvious or unique
• Consider the 8-puzzle

13
Possible Evaluation Functions for the 8-Puzzle

• Count how many tiles are out of place
• The state with the fewest tiles out of place may
  be best
• Sum the distances by which all tiles are out of
  place, counting one for each square an
  out-of-place tile must move
• This may be viewed as an improvement, because now
  we are considering how out of place each tile is
• Multiply 2 times the number of tiles in adjacent,
  but opposite, positions to those they should have
  in the goal state
• This may be viewed as an improvement, because it
  is hard to reverse two tiles that are right next
  to each other
• Add the sum of the distances to 2 times the
  number of direct reversals
• How do we know which to use?
• Run empirical tests to see which gives the best
  results

14
Sample Evaluation of Heuristic Functions
15
The Form of Heuristic Evaluation Functions

• Heuristic evaluation functions are heavily used
  in game playing and theorem proving
• Many evaluation functions take the form
• f(n) g(n) h(n)
• where
• n is any state (or node)
• g(n) is the actual length of
  the path from the root to n
• (This is known!)
• h(n) is the best guess of the
  path length from n to a solution
• Taking the distance traveled so far into account
  helps in finding the shortest path from the root
  to a goal

16
Search with h(n) the Number of Tiles out of Place
17
Heuristic Search and Expert Systems

• In expert systems, and other complex AI
  applications, you typically need more than one
  heuristic function to guide search
• Each subproblem may require its own heuristics
• The rules of thumb in an expert system may
  themselves be considered as heuristics
• Confidence measures are sometimes added to rules
  to indicate how likely it is that the heuristic
  is a good one
• Example from the financial advisor

18
Evaluating Heuristics

• One desirable property of a search algorithm is
  admissibility
• A search algorithm is admissible if it is
  guaranteed to find the shortest path to a goal
  whenever one exists
• Ex Breadth-first search is admissible, because
  it examines every node at higher levels before
  going on to those at lower levels
• We define a function f(n) g(n) h(n) where
• g(n) is the cost of the shortest
  path from the root to node n
• h(n) is the actual cost of the
  shortest path from n to the goal
• which makes
• f(n) the actual cost of the optimal
  path from the root to a goal that
• goes through node n
• f(n) is sometimes called an oracle
• Unfortunately, oracles seldom exist
• However, we try to define an f(n) that is close
  to f(n)

19
Algorithm A, Admissibility, Algorithm A
20
Examples of Admissible Algorithms

• Breadth-first search is an admissible algorithm
  that uses the function
• f(n) g(n) 0
• It only considers the g(n), the distance traveled
  so far, in determining which node to consider
  next
• Note that 0 is guaranteed to be equal or less
  than the actual cost of the path from node n to a
  goal
• Our 8-puzzle heuristics are also admissible
• We know that the number of tiles out of place is
  less than or equal to the number of moves it will
  take to get them into place
• The sum of the distances of how far out of place
  tiles are is also less than or equal to the
  number of moves it will take to get them into
  place
• 2 the number of direct tile reversals is also
  less than the number of moves needed to make the
  reversals, let alone to get all of the other
  tiles in place

21
Monotonicity

• The A algorithm is admissible, but it may take
  non-optimal paths to non-goal states along the
  way
• An algorithm that consistently takes the optimal
  path to each state it visits is monotonic, as
  well as admissible
• When best-first search has a monotonic heuristic,
  it can skip the step of rechecking nodes to see
  if they have been reached by shorter paths
• Note that breadth-first search is monotonic as
  well as admissible.

22
Informedness

• For two A heuristics, h1 and h2, if h1(n) lt
  h2(n), for all states n, then h2 is more informed
  than h1
• A better informed heuristic will expand less of
  the search space to find the optimal path to a
  goal
• Example for the 8-puzzle Compare h1,
  breadth-first search, to h2, best-first search
  with the heuristic of counting tiles out of place
• For breadth-first search, h1(n) is always 0
• h2(n) is 0 when the goal is reached, and
  otherwise, it is gt 0
• So, h1(n) lt h2(n), and best-first search is more
  informed than breadth-first search
• Once you have a heuristic that works, you can
  try to find a better informed one
• Theres a trade-off, in that calculating the
  value for a complex heuristic function may use as
  much or more time than searching more nodes

23
Game Playing Heuristics

• Minimax is a heuristic for two-player games,
  which can be used whether or not the state space
  can be exhaustively searched
• In a simple game, like nim, you can map out the
  whole search space
• In nim, a pile of tokens is placed on a table
• Players take turns dividing a pile into two piles
• It is not legal to divide a pile evenly
• The player who can not make a move (because there
  is no pile left that can be divided into two
  uneven piles) loses
• The two opponents are called Min and Max. We
  assume that Max is trying to win by maximizing a
  score, and that Min is trying to make Max lose,
  by minimizing the score
• The levels of the graph are labeled by which
  player must move next
• A leaf node is labeled 1 if it is a win for Max,
  0 if it is a loss
• The value of each node higher up in the graph is
  the maximum value of its children (for Max) and
  the minimum of its children (for Min)

24
A Minimax State Space for Nim
25
Minimaxing to a Fixed Ply Depth

• When the state space can not be exhaustively
  searched in a reasonable amount of time, you can
  minimax to a fixed ply depth
• A ply is just a level of the graph
• Since you dont have all of the leaf nodes, you
  dont have absolute values to percolate up the
  graph
• So you need a heuristic to tell how likely a
  player is to win
• Example In tic tac toe, you could see how many
  possibilities there are for each player to get
  three in a row
• Then Max would choose so as to maximize the
  difference in his favor and Min would choose to
  minimize that difference

26
The Heuristic for Tic Tac Toe
27
Two-Ply Minimax in Tic Tac Toe
28
Alpha-Beta Pruning

• Alpha-beta pruning can be used with minimax to
  make it more efficient
• Alpha-beta search proceeds in depth-first fashion
  to the maximum ply
• Max nodes are given alpha values, which can not
  decrease, and Min nodes are given beta values,
  which can not increase
• If the lowest level is a Min level, the maximum
  value of the children at that level is passed up
  to the parent, at the Max level, as an alpha
  value
• Then, the grandparent, at the Min level, can use
  that value as a beta cutoff
• There are two rules for terminating search in a
  direction
• Search can be stopped below a Min node with a
  beta value lt the alpha value of any of its Max
  ancestors
• Search can be stopped below a Max node with an
  alpha value gt the beta value of any of its Min
  ancestors
• In the best case, alpha-beta pruning may be able
  to search twice as deep as minimax, in the same
  time
• In the worst case, it searches no more of the
  state space than minimax

29
Example of Minimax without Alpha-Beta Pruning
30
The Same Example, with Alpha-Beta Pruning