Notes 3: Extensions of Blind Search

1
Notes 3 Extensions of Blind Search

• ICS 171 Summer 1999

2
Summary

• Search basics
• search spaces and search trees
• complexity of search algorithms
• Some new search strategies
• iterative deepening
• bidirectional search
• Repeated states can lead to infinitely large
  search trees
• methods for for detecting repeated states
• But all of these are blind algorithms in the
  sense that they do not take into account how far
  away the goal is.

3
Defining Search Problems

• A statement of a Search problem has 4 components
• 1. A set of states
• 2. A set of operators which allow one to get
  from one state to another
• 3. A start state S
• 4. A set of possible goal states, or ways to test
  for goal states
• Search solution consists of
• a unique goal state G
• a sequence of operators which transform S into a
  goal state G
• For now we are only interested in finding any
  path from S to G

4
Important A State Space and a Search Tree are
different
S
B
D
S
B
C
C
B
State Space
Example of a Search Tree

• A State Space represents all states and operators
  for the problem
• A Search Tree is what an algorithm constructs as
  it solves a search problem
• so we can have different search trees for the
  same problem
• search trees grow in a dynamic fashion until the
  goal is found

5
Why is Search often difficult?

• At the start of the search, the search algorithm
  does not know
• the size of the tree
• the shape of the tree
• the depth of the goal states
• How big can a search tree be?
• say there is a constant branching factor b
• and a goal exists at depth d
• search tree which includes a goal can have
  bd different branches in the tree
• Examples
• b 2, d 10 bd 210 1024
• b 10, d 10 bd 1010 10,000,000,000

6
Quick Review of Complexity

• In analyzing an algorithm we often look at
  worst-case
• 1. Time complexity
• (how many seconds it will take to run)
• 2.. Space complexity
• (how much memory is required)
• The complexity will be a function of the size of
  the inputs to the algorithm, e.g., n is the
  length of a list to be sorted
• we want to know how the algorithm scales with n
• We use notation O( f(n) ) to denote worst-case
  behavior, e.g.,
• O(n) for linear behavior
• O(n2) for quadratic behavior
• O(cn) for exponential behavior, c is some
  constant
• In practice we want algorithms which scale
  nicely

7
What is the Complexity of Depth-First Search?

• Time Complexity
• assume (worst case) that there is 1 goal leaf at
  the RHS
• so DFS will expand all nodes 1 b b2
  ......... bd O (bd)
• Space Complexity
• how many nodes can be in the queue (worst-case)?
• at depth l lt d we have b-1 nodes
• at depth d we have d nodes
• total (d-1)(b-1) d O(bd)

d0
d1
d2
G
d0
d1
d2 d3 d4

8
What is the Complexity of Breadth-First Search?

• Time Complexity
• assume (worst case) that there is 1 goal leaf at
  the RHS
• so BFS will expand all nodes 1 b b2
  ......... bd O (bd)
• Space Complexity
• how many nodes can be in the queue (worst-case)?
• at depth d-1 there are bd unexpanded nodes in
  the Q O (bd)

d0
d1
d2
G
d0
d1
d2
G
9
Examples of Time and Memory Requirements for
Breadth-First Search
Depth of Nodes Solution Expanded Time Memory
0 1 1 millisecond 100 bytes 2 111 0.1
seconds 11 kbytes 4 11,111 11 seconds 1
megabyte 8 108 31 hours 11 giabytes 12 1012
35 years 111 terabytes
Assuming b10, 1000 nodes/sec, 100 bytes/node
10
Comparing DFS and BFS

• Same Time Complexity, unless...
• say we have a search problem with
• goals at some depth d
• but paths without goals and which have infinite
  depth (i.e., loops in the search space)
• in this case DFS never may never find a goal!
• (it stays on an infinite (non-goal) path forever)
• BFS does not have this problem
• it will find the finite depth goals in time
  O(bd)
• Practical considerations
• if there are no infinite paths, and many possible
  goals in the search tree, DFS will work best
• For large branching factors b, BFS may run out of
  memory
• BFS is safer if we know there can be loops

11
Search Method 3 Depth-Limited Search

• This is Depth-first Search with a cutoff on the
  maximum depth of any path
• i.e., implement the usual DFS algorithm
• when any path gets to be of length m, then do not
  expand this path any further and backup
• this will systematically explore a search tree of
  depth m
• Properties of DLS
• Time complexity O(bm), Space complexity
  O(bm)
• If goal state is within m steps from S
• DLS is complete
• e.g., with N cities, we know that if there is a
  path to goal state G it can be of length N-1 at
  most
• But usually we dont know where the goal is!
• if goal state is more than m steps from S, DLS is
  incomplete!
• gt the big problem is how to choose the value of m

12
Search Method 4 Iterative Deepening Search

• Basic Idea
• we can run DFS with a maximum depth constraint, m
• i.e., DFS algorithm but it backs-up at depth m
• this avoids the problem of infinite paths
• But how do we choose m in practice? say m lt d
  (!!)
• We can run DFS multiple times, gradually
  increasing m
• this is known as Iterative Deepening Search

Procedure for m 1 to infinity if (depth-first
search with max-depth m ) returns
success then report (success) and
quit else continue end
13
Comments on Iterative Deepening Search

• Complexity
• Space complexity O(bd)
• (since its like depth first search run different
  times)
• Time Complexity
• 1 (1b) (1 bb2) .......(1 b....bd)
  O(bd) (i.e., the same as BFS or DFS in the the
  worst case)
• The overhead in repeated searching of the same
  subtrees is small relative to the overall time
• e.g., for b10, only takes about 11 more time
  than DFS
• A useful practical method
• combines
• guarantee of finding a solution if one exists (as
  in BFS)
• space efficiency, O(bd) of DFS

14
An Aside Non-deterministic Search

• All of the search methods we have discussed are
  deterministic
• deterministic no randomness
• non-deterministic contains some random element
• How can a search algorithm contain randomness?
• select nodes for expansion (from the Q) at random
• this can avoid infinite depth problems
• we will return later to look at some algorithms
  which introduce randomness on purpose - strange
  idea at first but it can help!
• Side-topic
• Can an algorithm really be random?

15
Search Method 5 Bidirectional Search

• Idea
• simultaneously search forward from S and
  backwards from G
• stop when both meet in the middle
• need to keep track of the intersection of 2 open
  sets of nodes
• What does searching backwards from G mean
• need a way to specify the predecessors of G
• this can be difficult,
• e.g., predecessors of checkmate in chess?
• what if there are multiple goal states?
• what if there is only a goal test, no explicit
  list?
• Complexity
• time complexity is O(2 b(d/2)) O(b (d/2)) steps
• memory complexity is the same

16
Repeated States
S
B
S
B
C
C
S
C
B
S
State Space
Example of a Search Tree

• For many problems we can have repeated states in
  the search tree
• i.e., the same state can be gotten to by
  different paths
• gt same state appears in multiple places in the
  tree
• this is inefficient, we want to avoid it
• How inefficient can this be?
• a problem with a finite number of states can have
  an infinite search tree!

17
Techniques for Avoiding Repeated States

• Method 1
• when expanding, do not allow return to parent
  state
• (but this will not avoid triangle loops for
  example)
• Method 2
• do not create paths containing cycles (loops)
• i.e., do not keep any child-node which is also an
  ancestor in the tree
• Method 3
• never generate a state generated before
• only method which is guaranteed to always avoid
  repeated states
• must keep track of all possible states (uses a
  lot of memory)
• e.g., 8-puzzle problem, we have 9! 362,880
  states
• Methods 1 and 2 are most practical, work well on
  most problems

18
Web Site of the Day

• Artificial Intelligence repository
• collection of summary articles, interviews,
  online software, demos, etc on the history of AI
• online proceedings available at
• http//hyperion.advanced.org/18242/index.shtml
• (you can also access this from the class Web
  page)
• browse around lots of interesting material on
  topics we will be covering in this course.

19
Summary

• A review of search
• a search space consists of states and operators
  it is a graph
• a search tree represents a particular exploration
  of search space
• There are various extensions to standard BFS and
  DFS
• depth-limited search
• iterative deepening
• bidirectional search
• Repeated states can lead to infinitely large
  search trees
• we looked at methods for for detecting repeated
  states
• Reading Chapter 3 page 73 to 82 (except for
  uniform-cost)
• All of the search techniques so far do not care
  about the cost of the path to the goal. Next we
  will look at algorithms which are optimal in the
  sense they always find a path to goal which has
  minimum cost.