Chapter 5 Backtracking

1
Chapter 5Backtracking

• 5.1 The Backtracking Technique
• 5.2 The n-Queens Problem
• 5.3 Using a Monte Carlo Algorithm to Estimate
• the Efficiency of a Backtracking Algorithm
• 5.4 The Sum-of-Subsets Problem
• 5.5 Graph Coloring
• 5.6 The Hamiltonian Circuits Problem
• 5.7 The 0-1 Knapsack Problem

2
5.1 Backtracking Technique

• Choose a sequence of objects from a specified
• set so that the sequence satisfies some
  criterion.
• 1. n-Queens Problem position n queens on an n?n
  chessboard so that no two queens threaten each
  other (no two queens can be in the same row,
  column/diagonal).
• 2. Depth-First Search find a spanning tree from
  a graph. A preorder tree traversal is a
  depth-first search of the tree. And backtracking
  is a modified depth-first search of a tree.

3
Depth-First Tree Search

• void depth_first_tree_search(node v)
• node u
• visit v
• for (each child u of v)
• depth_first_tree_search(u)

4
n-Queens Problem

• void checknode(node v)
• node u
• if (promising(v))
• if (there is a solution at v)
• write the solution
• else (each child u of v)
• for (each child u of v)
  
• checknode(u)

5
The Backtracking Technique

• The procedure whereby, after determining that
  a node lead to nothing but dead ends, we go back
  (backtrack) to the nodes parent and proceed
  with the search on the next child.
• Promisingthe node can lead to a solution,
  otherwise, it is called as nonpromising.
• Pruningcheck each node whether it is promising,
  if not, backtracking to the nodes parent.
• Backtracking is the procedure to prune state
• space tree.

6
Example 5.14-Queens Problem
?
?
?
?
7
n-Queens Problem

• void expand(node v)
• node u
• for (each child u of v)
• if (promising(v))
• if (there is a solution at v)
• write the solution
• else
  
• expand(u)

8
5.2 n-Queens Problem
Criterion In the chessboard, two queens can not
be in the same row, column or diagonal. col(i)
the column where the queen is in the ith row. In
the same column col(i) col(k)
In the diagonal
col(i) ? col(k) i ? k or col(i) ?
col(k) k ? i
9
Algorithm 5.1 n-Queens Problem

• ProblemPosition n queens on a chessboard so that
  no
• two are in the same row,
  column, or diagonal.

Inputspositive integer n. Outputall possible
ways n queens can be placed.
void queens (int i) index j if
(promising(i)) if (in)
cout ltlt col1 through coln
else for ( j 1 j lt n j)
coli1 j
queensi1
void promising (int i) index k
bool switch k 1
switch true while ( k lt i
switch) if (colicolk
abs(coli?colk) i?k)
switch false
k
return switch
10
Analysis of Algorithm 5.1

• Worst-Case Time Complexity Analysis
• Number of nodes in the entire state space
  tree
• A more tight bound

n No. of Nodes Checked by Algo. 1 (DFS) No. of Candidate Solutions Checked by Algo. 2 (n!) No. of Nodes Checked by Backtracking No. of Nodes found promising by Backtracking
4 8 12 14 341 19173961 9.73?1012 1.20?1016 24 40320 4.79?108 8.72?1010 61 15721 1.01?107 3.78?108 17 2057 8.56?105 2.74?107
11
5.3 Use a Monte Carlo Algorithm to Estimate the
Efficiency of a Backtracking Algorithm

1. Monte Carlo algorithms are probabilistic
   algorithms.
2. Unless otherwise stated, we assume the
   probability distribution is the uniform
   distribution.
3. A Monte Carlo algorithm estimates the expected
   value of a random variable, defined on a sample
   space.
4. No guarantee the estimate is closed to the true
   expected value, but the probability that it is
   close increases as the time available to the
   algorithm increases.
5. Generate a typical path in the tree consisting
   of the nodes the would be checked, then estimate
   the number of nodes in this tree from the path.

12
Monte Carlo Algorithm

• Let m0 be the number of promising children of the
  root.
• Randomly generate a promising node at level 1.
  Let m1 be the number of promising children of
  this node.
• Randomly generate a promising child of the node
  obtained in the previous step. Let m2 be the
  number of promising children of this node.
• 4. Randomly generate a promising child of the
  node obtained in the previous step. Let mi be the
  number of promising children of this node.

13
Algorithm 5.2Monte Carlo Estimate

• ProblemEstimate the efficiency of a backtracking
  
• algorithm using a Monte Carlo
  algorithm.

Inputsan instance of the problem that the
backtracking algorithm solves.
Outputan estimate of the number of nodes in the
pruned state space tree produced by the
algorithm.
int estimate ( ) node v int m, mprod,
t, numnodes v root of state space tree
numnodes 1 m 1 while ( m! 0)
t number of children of v
mprod mprodm
numnodes numnodes mprodt
m number of promising
children of v
if ( m! 0)
v randomly selected
promising child of v
return numnodes

14
Algorithm 5.3Estimate for Algorithm 5.1

• ProblemEstimate the efficiency of Algorithm 5.1.
  

Inputspositive integer n. Outputan estimate of
the number of nodes in the pruned state
space tree produced by the algorithm.
int estimate_n_queens (int n) index i, j,
col1..n int m, mprod, numnodes
set_of_index prom_children i 0
numnodes 1 m 1 while ( m ! 0
i ! n ) mprod mprodm
numnodes mprodt i
m 0 prom_children ?
for (j 1 j lt n j)
coli j if (
promising(i)) m
prom_children
prom_children ?j

if ( m! 0) j randomly
selection from
prom_children
coli j
return numnodes
15
5.4 The Sum-of-Subsets Problem
In the knapsack problem, if the profit of each
item is the same, then the goal is to maximize
the total weight while not exceed W. So the
thief might first try to determine whether there
was a set whose total weight equaled W. The
problem of determining such sets is called the
Sum-of-Subsets Problem.
16
Example 5.2
Suppose that n 5, W 21, and w1 5,
w2 6, w3 10, w4 11, and w5 16. Find
the solutions. Sol
17
Example 5.3
Suppose that n 3, W 6, and w1 2, w2
4, w3 5. Find the solutions. Sol
18
Example 5.4
Suppose that n 4, W 13, and w1 3,
w2 4, w3 5 , w4 6. Find the
solutions. SolFor the weights sorting in
nondecreasing order, a node is nonpromising if
weightwi1gtW where weight is the total weight up
to a node at level i. weighttotalrltW is also
nonpromising.
19
Algorithm 5.4

• ProblemGiven n positive weights and a positive
  integer
• W, find all combinations of the weights
  that sum to W.

Inputspositive integer n, sorted array w index
from 1 to n, and a positive integer
W. Output all combinations of the weights that
sum to W.
void sum_of_subsets (index i, int weight, int
total) if (promising(i)) if
(weight W) cout ltlt include1
through includei else
includei1 yes
sum_of_subsets(i1, weightwi1,
total?wi1) includei1
no sum_of_subsets(i1,
weight, total?wi1) bool
promising (index i) return
(weighttotal gt W) (weight W
weightwi1ltW)

20
Analysis of Algorithm 5.4
Worst-Case Time Complexity Number of nodes in
the state space tree searched If it needs an
exponentially large number of nodes to be
visited.
21
5.5 Graph Coloring
?m-Coloring Problem Find all ways to color an
undirected graph using at most m color, so that
no two adjacent vertices are the same color.
22
Example 5.5
Find all ways to color the 4 vertices graph. Sol
If m 2, no solution. If m 3,
(v1 color 1), (v2, v4 color 2), (v3 color 3)
(v1 color 1), (v2, v4
color 3), (v3 color 2)
(v1 color 2), (v2, v4 color 1), (v3 color
3) (v1 color 2),
(v2, v4 color 3), (v3 color 1)
(v1 color 3), (v2, v4 color 1),
(v3 color 2) (v1
color 3), (v2, v4 color 2), (v3 color 1).
If m 4, (vi color i), for i 1 to 4,
total 4! 24. So what we concerned is the
case m lt N, the number of
vertices.
23
Algorithm 5.5

• ProblemFind all ways for the m-Coloring Problem.
  

Inputspositive integer n, m, and an adjacent
matrix W. Output an array vcolor, where
vcolori is the color of vertex
i.
bool promising(index i) index j bool
switch switch true j 1 while
(j lt i switch) if (Wij
vcolorivcolorj)
switch false
j return switch

• void m_coloring (index i)
• int color
• if (promising(i))
• if (in)
• cout ltlt vcolor1 through vcolorn
• else
• for (color1 colorltm color)
  
• vcolori1color
• m_coloring(i1)

24
Analysis of Algorithm 5.5
Worst-Case Time Complexity Number of nodes in
the state space tree searched
25
5.6 The Hamiltonian Circuits Problem
Hamiltonian Circuit (Tour) A path that
starts at a given vertex, visits each vertex in
the graph exactly once, and ends at the starting
vertex.
26
State Space Tree
Put the starting vertex at level 0 in the tree
call it the zeroth vertex on the path. At level
1, consider each vertex other than the starting
vertex as the first vertex after the starting
one. At level 2, consider each of these same
vertices as the second vertex, and so on.
Finally, at level n-1, consider each of these
same vertices as the (n-1)st vertex.

27
Backtrack in the State Space Tree
1. The ith vertex on the path must be adjacent to
the (i?1)st vertex on the path. 2. The (n?1)st
vertex must be adjacent to the 0th vertex (the
starting one). 3. The ith vertex cannot be one of
the first (i?1) vertices.
28
Algorithm 5.6

• ProblemFind all Hamiltonian Circuits for the
  graph.

Inputsan undirected graph with n vertices, and
an adjacency matrix W. Output an
array vindex, where vindexi is the index of
the i-th vertex on the path.

• void hamiltonian (index i)
• index j
• if (promising(i))
• if (I n?1)
• cout ltlt vindex0 through
• vindexn?1
• else
• for (j 2 j lt n j)
• vindexi1 j
• hamiltonian(i1)

bool promising(index i) index j bool
switch if (in?1 ! Wvindexn?1vindex0
) switch false else if (igt0
! Wvindexi?1vindexi) switch
false else switch true
j 1 while (j lt i switch)
if (vindexi vindexj)
switch false
j return switch
29
Analysis of Algorithm 5.6
Worst-Case Time Complexity Number of nodes in
the state space tree searched
30
5.7 The 0-1 Knapsack Problem
Backtracking for the 0-1 Knapsack Problem void
checknode (node v) node u if
(value(v) is better than best) best
value(v) if (promising(v)) for
(each child u of v)
checknode(u)
31
Nonpromising Nodes
weightthe sum of the weights of the items that
have been included up to some node.
? weight ? W profitthe sum of the profits
of the items included up to some node.
Initialization bound profit, totweight
weight
If bound ? maxprofit

? Nonpromising
32
Example 5.6

• Suppose that n 4, W16, and we have the
  following
• i pi wi pi/wi
• 1 40 2 20
• 2 30 5 6
• 3 50 10 5
• 4 10 5 2
• Find the solutions.
• Sol
• Set maxprofit 0
• Visit node(0,0)
• profit ?
• weight ?
• 3. Visit node (1,1) profit ? weight
  ?

bound ?
maxprofit ?
bound ?
33
Example 5.6 (Contd)

• n 4, W16,
• 4. Visit node(2,1)
• profit ?
• weight ?
• 5. Visit node (3,1)

maxprofit ?
bound ?
34
Algorithm 5.7

• ProblemGiven n positive weights and a positive
  integer
• W, find all combinations of the weights
  that sum to W.

Inputspositive integer n, sorted array w index
from 1 to n, and a positive integer
W. Output all combinations of the weights that
sum to W.
void knapsack (index i, int profit, int
weight) if (weight lt W profit gt
maxprofit) maxprofit profit
numbest i bestset
include if (promising(i))
includei1 yes knapsack(i1,
profitpi1, weightwi1)
includei1 no knapsack(i1,
profit, weight)

35
Algorithm 5.7
bool promising (index i) index j, k
int totweight float bound if (weight
gt W ) return false else j i
1 bound profit
totweight weight while (j lt n
totweightwj lt W)
totweight totweight wj
bound bound pj j
k j if (k
lt n) bound bound (W?
totweight)pk/wk return bound
gt maxprofit