Problem Solving through Exhaustive State Space Search

1
Problem Solving throughExhaustive State Space
Search

• Jacques Robin

2
Outline

• Search Agent
• Formulating Decision Problems as Navigating State
  Space to Find Goal State
• Generic Search Algorithm
• Specific Algorithms
• Breadth-First Search
• Uniform Cost Search
• Depth-First Search
• Backtracking Search
• Iterative Deepening Search
• Bi-Directional Search
• Comparative Table
• Limitations and Difficulties
• Repeated States
• Partial Information

3
Search Agents

• Generic decision problem to be solved by an
  agent
• Among all possible action sequences that I can
  execute,
• which ones will result in changing the
  environment from its current state,
• to another state that matches my goal?
• Additional optimization problem
• Among those action sequences that will change
  the environment from its current state to a goal
  state
• which one can be executed at minimum cost?
• or which one leads to a state with maximum
  utility?
• Search agent solves this decision problem by a
  generate and test approach
• Given the environment model,
• generate one by one (all) the possible states
  (the state space) of the environment reachable
  through all possible action sequences from the
  current state,
• test each generated state to determine whether
  it satisfies the goal or maximize utility
• Navigation metaphor
• The order of state generation is viewed as
  navigating the entire environment state space

4
Example of Natural Born Search Problem
5
Example of Applying Navigation Metaphorto
Arbitrary Decision Problem

• N-Queens problem how to arrange n queens on a
  NxN chess board in a pacific configuration where
  no queen can attack another?

• Full-State Formulation local navigation in
  thefull-State Space

• Partial-State Formulation global navigation in
  partial-State Space

6
Search Problem Taxonomy
Search Problem
7
Search Tree

• The state space can be represented as a search
  tree
• Each search tree node represents an environment
  state
• Each search tree arc represents an action
  changing the environment from its source node to
  its target node
• Each node or arc can be associated to a utility
  or cost
• Each path from the tree root to another tree node
  represents an action sequence
• Some search tree node leaves represent goal
  states
• The problem of navigating the state space then
  becomes one of generating and maintaining a
  search tree until a solution node or path is
  generated
• The problem search tree
• An abstract concept that contains one node for
  each possible environment state
• Can be infinite
• The algorithm search tree
• A concrete data structure
• A subset of the problem search tree that
  contains only the nodes generated and maintained
  (i.e., explored) at the current point during the
  algorithm execution (always finite)
• Problem search tree branching factor average
  number of actions available to the agent in each
  state
• Algorithm effective branching factor average
  number of nodes effectively generated as
  successors of each node during search

8
Search Tree Example Vacuum Cleaner World
u
u
u
u
s
s
s
u
d
u
d
u
d
u
s
s
s
d
s
s
d
d
d
d
u
s
d
u
d
s
u
d
s
u
d
s
9
Search Methods

• Searching the entire space of the action sequence
  reachable environment states is called exhaustive
  search, systematic search, blind search or
  uninformed search
• Searching a restricted subset of that state space
  based on knowledge about the specific
  characteristics of the problem or problem class
  is called partial search
• A heuristic is an insight or approximate
  knowledge about a problem class or problem class
  family on which a search algorithm can rely to
  improve its run time and/or space requirement
• An ordering heuristic defines
• In which order to generate the problem search
  tree nodes,
• When to go next while navigating the state space
  (to get closer faster to a solution point)
• A pruning heuristic defines
• Which branches of the problem search tree to
  avoid generating alltogether,
• What subspaces of the state space not to explore
  (for they cannot or are very unlikely to contain
  a solution point)
• Non-heuristic, exhaustive search is not scalable
  to large problem instances (worst-case
  exponential in time and/or space)
• Heuristic, partial search offers no warrantee to
  find a solution if one exist, or to find the best
  solution if several exists.

10
Formulating a Agent Decision Problemas a Search
Problem

• Define abstract format of a generic environment
  state, ex., a class C
• Define the initial state, ex., a specific object
  of class C
• Define the successor operation
• Takes as input a state or state set S and an
  action A
• Returns the state or state set R resulting from
  the agent executing A in S
• Together, these 3 elements constitute an
  intentional representation of the state space
• The search algorithm transforms this intentional
  representation into an extensional one, by
  repeatedly applying the successor operation
  starting from the initial state
• Define a boolean operation testing whether a
  state is a goal, ex., a method of C
• For optimization problems additionally define a
  operation that returns the cost or utility of an
  action or state

11
Problem Formulation Most Crucial Factorof Search
Efficiency

• Problem formulation is more crucial than choice
  of search algorithm or choice of heuristics to
  make an agent decision problem effectively
  solvable by state space search
• 8-queens problem formulation 1
• Initial state empty board
• Action pick column and line of one queen
• Branching factor 64
• State-space 864
• 8-queens problem formulation 2
• Initial state empty board
• Action pre-assign one column per queen, pick
  only line in pre-assigned column
• Branching factor 8
• State-space 88

12
Generic Exhaustive Search Algorithm

• Initialize the fringe to the root node
  representing the initial state
• Until goal node found in fringe, repeat
• Choose one node from fringe to expand by calling
  its successor operation
• Extend the current fringe with the nodes
  generated by this successor operation
• If optimization problem, update path cost or
  utility value
• Return goal node or path from root node to goal
  node
• Specific algorithms differ in terms of the order
  in which they respectively expand the fringe
  nodes

Arad
Sibiu
Timisoara
Zenrid
Arad
Fagaras
Oradea
R.Vilcea
Arad
Lugoj
Arad
Oradea
13
Generic Exhaustive Search Algorithm

• Initialize the fringe to the root node
  representing the initial state
• Until goal node found in fringe, repeat
• Choose one node from fringe to expand by calling
  its successor operation
• Extend the current fringe with the nodes
  generated by this successor operation
• If optimization problem, update path cost or
  utility value
• Return goal node or path from root node to goal
  node
• Specific algorithms differ in terms of the order
  in which they respectively expand the fringe
  nodes

Arad
fringe
Sibiu
Timisoara
Zenrid
Arad
Fagaras
Oradea
R.Vilcea
Arad
Lugoj
Arad
Oradea
14
Generic Exhaustive Search Algorithm

• Initialize the fringe to the root node
  representing the initial state
• Until goal node found in fringe, repeat
• Choose one node from fringe to expand by calling
  its successor operation
• Extend the current fringe with the nodes
  generated by this successor operation
• If optimization problem, update path cost or
  utility value
• Return goal node or path from root node to goal
  node
• Specific algorithms differ in terms of the order
  in which they respectively expand the fringe nodes

Arad
open-list
Sibiu
Timisoara
Zenrid
fringe
Arad
Fagaras
Oradea
R.Vilcea
Arad
Lugoj
Arad
Oradea
15
Search Algorithms Characteristics and Performance

• Complete guaranteed to find a solution if one
  exists
• Optimal (for optimization problem) guaranteed to
  find the best (highest utility or lowest cost)
  solution if one exists
• Input parameters to complexity metrics
• b problem search tree branching factor
• d depth of highest solution (or best solution
  for optimization problems) in problem search tree
• m problem search tree depth (can be infinite)
• Complexity metrics of algorithms
• TimeComplexity(b,d,m) number of expanded nodes
• SpaceComplexity(b,d,m) maximum number of nodes
  needed in memory at one point during the
  execution of the algorithm

16
Exhaustive Search Algorithms

• Breadth-First Expand first most shallow node
  from fringe
• Uniform Cost Expand first node from fringe of
  lowest cost (or highest utility) path from the
  root node
• Depth-First Expand first deepest node from
  fringe
• Backtracking Depth first variant with fringe
  limited to a single node
• Depth-Limited Depth-first stopping at depth
  limit N.
• Iterative Deepening Sequence of depth limited
  search at increasing depth
• Bi-directional
• Parallel search from initial state and from goal
  state
• Solution found when the two paths under
  construction intersect

17
Breadth-First Search
Fringe
A
B
C
E
F
D
G
K
M
I
O
J
L
H
N
18
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
19
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
20
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
21
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
22
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
23
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
24
Breadth-First Search
Fringe
A
Expanded
B
C
E
F
D
G
K
M
I
O
J
L
H
N
25
Uniform Cost Search
B
1
10
5
5
Problem Graph
C
E
A
5
15
D
A
26
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
27
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
28
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
29
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
30
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
31
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
32
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
33
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
34
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
35
Backtracking Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
36
Backtracking Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
37
Depth-First Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
38
Backtracking Search
A
B
C
E
F
D
G
J
L
H
N
K
M
I
O
39
Backtracking Search
A
B
C
E
F
D
G
J
L
N
K
M
I
O
40
Backtracking Search
A
B
C
E
F
G
J
L
N
K
M
O
41
Backtracking Search
A
B
C
E
F
G
J
L
N
K
M
O
42
Backtracking Search
A
B
C
E
F
G
L
N
K
M
O
43
Backtracking Search
A
C
F
G
L
N
M
O
44
Iterative Deepening
A
45
Bi-Directional Search

• Two parallel searches one from the current state
  and one from the goal state
• When they reach a common node a path from
  current to goal has been found
• Time complexity halved O(bd/2) O(bd/2)
  O(bd/2) ltlt O(bd)
• But not alwayspossible
• Irreversibleactions
• Large numberof intentionallyspecifiedgoal
  states

46
Comparing Search Algorithms
Breadth-First Uniform-Cost Depth-First Back-tracking Iterative Deepening Bi-Directional
Complete if b finite if all step costs positives no no if b finite depends on search used on each direction
Optimal if all steps share same cost yes no no if all steps share same cost depends on search used on each direction
Time Complexity O(bd1) O(b?C/e?) O(bm) O(bm) O(bd) O(bd/2)
Space Complexity O(bd1) O(b?C/e?) O(b.m) O(m) O(b.d) O(bd/2)

• C cost of optimal solution
• ?a ? actions(agent), a ? e

47
Searching Environmentswith Repeated States

• In some search problems,
• each environment state is only reachable through
  a single action sequence
• ex, such as the N-Queens problem in the
  pre-assigned column formulation,
• However, for most problems,
• the same environment state can be reached through
  several, many or even an infinity of distinct
  action sequences,
• This leads to the repeated generation of the same
  nodes in different branches of the search tree

saa
sab
sbb
szb
saa
sba
sza
sbz
szz
saz
48
Effects of Repeated States onExhaustive Search
Algorithms

• They must be modified to include test comparing
  newly generated nodes with already generated ones
  and prune the repeated ones from the frontier
• Without such test
• Breadth-first and uniform-cost search fill
  memory far sooner with repeated nodes and likely
  before reaching the depth of the first goal node
• Depth-first and backtracking search can enter in
  deepening loop, never returning even in the
  presence of shallow goal nodes
• With such test
• Depth-first, backtracking and iterative
  deepening search loose their linear worst-case
  space complexity,
• for any guarantee to avoid all loops may require
  keeping an exponential number of expanded nodes
  in memory,
• in practice, the probability of loop occurrence
  is traded-off for size of the expanded node
  history
• Iterative deepening is no longer guaranteed to
  find first the optimal solution (lowest-cost
  path) for it may go deep to find a longer path
  before backing up and find a shorter one

49
In which Environments can a State Space Searching
Agent (S3A) Act Successfully?

• Fully observable? Partially observable?
  Sensorless?
• An S3A can act successfully in an environment
  that is either partially observable or even
  sensorless, provided that it is deterministic,
  small, lowly diverse and that the agent possesses
  an action model
• Instead of searching in the space of states, it
  can search in the space of possible state sets
  (beliefs) to know which actions are potentially
  available
• Deterministic or stochastic?
• An S3A can act successfully in an environment
  that is stochastic provided that it is small and
  at least partially observable
• Instead of searching offline in the space of
  states, it can search offline in the space of
  possible state sets (beliefs) resulting of an
  action with stochastic effects
• It can also search online and generate action
  programs or conditional plans instead of mere
  action sequences, ex, suck, right, if
  right.dirty then suck
• Discrete or continuous?
• An S3A using a global search algorithm can only
  act successfully in an discrete environment, for
  continuous variables imply infinite branching
  factor

50
In which Environments can a State Space Searching
Agent (S3A) Act Successfully?

• Episodic or non-episodic?
• An S3A is only necessary for a non-episodic
  environment which requires choosing action
  sequences, where the choice of one action
  conditions the action range subsequently
  available
• Static? sequential? concurrent synchronous or
  concurrent asynchronous?
• An S3A can only act successfully in a static or
  sequential environment, for there is no point in
  searching for goal-achieving or optimal action
  sequences in an environment that changes due to
  factors outside the control of the agent
• Small or large?
• An S3A can only act successfully in a
  medium-sized environment provided that it is
  deterministic and at least partially observable,
  or small environment that are either stochastic
  or sensorless
• Lowly or highly diverse?
• An S3A can only act successfully in low
  diversity environment for diversity affects
  directly the branching factor of the problem
  search tree space

51
Searching in the Space of Possible State Sets