Counting CSP Solutions Using Generalized XOR Constraints

1
Counting CSP Solutions Using Generalized XOR
Constraints

• Carla P. Gomes, Willem-Jan van Hoeve,Ashish
  Sabharwal, Bart Selman
• Cornell University
• AAAI Conference, 2007
• Vancouver, BC

2
Problem Description

• Constraint Satisfaction Problem (CSP) P
• Input
• a set V of variables
• a set of corresponding domains of variable
  values finite
• a set of constraints on V
• Output
• a solution, i.e. an assignment of values to
  variables in V such that all constraints are
  satisfied
• Solution Counting how many solutions does P
  have?
• P-complete problem
• Many applications to probabilistic
  reasoning,adversarial reasoning (bounded
  length), etc.

Roth 96, Littman et al. 01, Sang et al. 04,
Darwiche 05, Domingos 06
3
Counting Techniques for CSPs

• Best known generic method exhaustive systematic
  search
• Continue branch-and-bound search after first
  solution found
• Simple implementation
• If terminates obtain exact solution count
• If out of time obtain lower bound
• Not as advanced as systematic counters for SAT
  Relsat, Cachet
• No problem division into components
• No formula caching / component caching
• No conflict-based learning
• Scalability issue

4
Specialized Counting Methods

• E.g. for Binary CSPs Angelsmark-Johnson 03,
  Kask-Dechter-Gogate 04
• Count blocks of solutions at a time
• Similar to SAT model counter called Relsat
• Limited by scalability issues of high granularity
  exact counters
• (counts often range in 1015 to 1060 and higher)
• For integer domains Morgado-Matos-Manqui
  nho-MarquesSilva 06
• Solution count preserving translation into
  pseudo-Boolean form
• Optional further translation into Boolean form
  (SAT) Bailleux-Boufkhad-Roussel 06
• Often better than direct integer domain counting
• Still limited scalability

5
This Work

• A generic technique for efficiently counting CSP
  solutions
• Provides lower and upper bounds on the solution
  count
• Provides a correctness confidence (e.g. a 99
  guarantee)
• Often very fast
• Builds upon the MBound framework for counting for
  SAT
• XOR-streamlining
  Gomes-Sabharwal-Selman AAAI-06
• Extends the idea to general CSPs by exploiting
• Richer structure generalized XORs
• Modular solution techniques specialized
  filtering / propagation

6
Background
7
What are XOR/Parity Constraints?

• Special constraints on Boolean variables
• a ? b ? c ? d 1 satisfied if an odd
  number of a,b,c,d are set to 1 e.g.
  (a,b,c,d) (1,1,1,0) satisfies it
  (1,1,1,1) does not
• b ? d ? e 0 satisfied if an even number
  of b,d,e are set to 1
• These translate into a small set of CNF clauses
• Used earlier in randomized reductions in
  Theoretical CSValiant-Vazirani 86

8
XORs for Counting SAT

• MBound approachchoose constraint X uniformly at
  random from all XOR constraints of size k (i.e.
  with k variables)
• Two crucial properties
• For any k, for every truth assignment A,Pr A
  satisfies X 0.5
• When kn/2, for every two truth assignments A and
  B,A satisfies X and B satisfies X are
  independent events(pairwise independence)

Good average behavior, some guarantees,lower
bounds
Provides low variation,stronger
guarantees,upper bounds
9
XORs for Counting SAT
solution count ? 2s?? or solution count ? 2s?
SATinstance
Off-the-shelfSAT Solver
s random XORconstraints
Iterate t times

• Key idea instead of modifying the solver for
  counting, modify the problem and feed
  to the usual solver
• Theorem Algorithm is correct with probability ?
  1 ? 2??t
• Gomes-Sabharwal-Selman AAAI-06Gomes-Hoffmann-S
  abharwal-Selman SAT-07

10
The Desired Effect
If each XOR cut the solution space roughly in
half, wouldget down to a unique solution in
roughly log2 M steps
11
Counting CSP Solutions
12
Counting CSP Solutions
We propose 3 XOR constraintsbased approaches
Individualfiltering
?
based on watched vars
Binary XORson shadow model
Globalfiltering
?
a ? b ? c 1 Lower and upper bounds
CSPinstance
based on Gaussian elim.
?
Individualfiltering
Generalized XORson CSP vars
based on dynamic prog.for knapsack constraints
a b c r (mod D) Lower bounds only
?
Global filter.
Inefficient, incomplete
13
CSP Counting Binary XORs
CSP binary shadowmodel
CSP
CSP Solver
solution count(bounds)
random binary XORson shadow vars
iterate

• Relatively simple idea
• Create a binary shadow model for the CSP
• For each (x, v ? Domain(x)), create a new
  variable yx,v ? 0,1
• Add constraint yx,v 1 iff x v
• Add random binary XORs over variables yx,v
• Apply Boolean XOR framework to obtain solution
  counts
• Same correctness guarantees as for SAT
• Often works quite well!

14
Filtering Binary XORs Individual
Variable assignment
Consider
c 0
a 1
a ? c ? d ? f 1
d 1
Unit propagation f 1(filter out 0 from
Domain(f))

• Filtering possible iff XOR constraint has
  exactly 1 unbound variable
• For efficiency, use watched variables technique
  from SAT
• Maintain watch on two yet unassigned variables
• Process constraint only when a watched variable
  becomes bound
• Find new variable to watch
• If no such variable possible, do unit propagation

15
Filtering Binary XORs Global
Consider the system of binary XOR constraints
a ? b ? c ? d 1 b ? c
? e 0 a ? d ? e 1

• Equivalent to a system of linear equations over
  F2
• Use Gaussian-elimination style filtering
  algorithm
• Diagonalize the matrix
• If a row has all 0s and r.h.s. 1, system
  unsatisfiable
• If a row has only one 1, assign value to variable
  based on r.h.s.
• Can prove achieves complete filtering
• each remaining free variable has support for both
  0 and 1

16
CSP Counting Generalized XORs

• No new binary shadow variables
• Instead, generalized XORs mod D directly on CSP
  vars a b d f g r (mod D),
  r ? 0, 1, , D-1

Max. domainsize
CSP Solver
CSP
solution count(bounds)
random generalizedXORs mod D
iterate
17
CSP Counting Generalized XORs

• Solution counting algorithm
• Independently repeat t times
• Add s random generalized XORs mod D to the
  CSP
• Solve streamlined CSP
• If all t iterations have solution, output
  num-solutions ? Ds??
• Else Fail
• Theorem For every CSP, Pr output is correct
  ? 1 ? D??t
• Proof sketch
• Compute expected number of surviving solutions
• Apply Markovs inequality and independence of
  iterations
• Note hybrid version add XORs, count remaining
  solutions, take min/avg

18
Filtering Generalized XORs

• Individual filtering for each generalized XOR
• User dynamic programming approach used for
  complete filtering of the knapsack constraint
  Trick 03
• E.g. a b c 1 (mod 3), a ? 0,1, b
  ? 1, c ? 0,1,2

2
2
2
Dont include edges that cannotbe reached from
the red node Remove edges that do notlead to
the green node Filter variable valuesE.g. c
cannot be 1
2
1
1
1
1
0
1
1
0
0
0
0
0
a
b
c
19
Counting CSP Solutions Recap.
3 XOR constraints based approaches
Individualfiltering
?
based on watched vars
Binary XORson shadow model
Globalfiltering
?
a ? b ? c 1 Lower and upper bounds
CSPinstance
based on Gaussian elim.
?
Individualfiltering
Generalized XORson CSP vars
based on dynamic prog.for knapsack constraints
a b c r (mod D) Lower bounds only
?
Global filter.
Inefficient, incomplete
20
Experimental Highlights, 1

• N-Queens problem
• order 20 order 30
• true count 3.9 x 1010 ---
• pure CSP search ? 3.5 x 106 ? 4.1 x 106 1 hour
• binary XORs, individual ? 1.1 x 106 ? 5.4 x 108
• binary XORs, global ? 2.1 x 106 ? 5.4 x
  108 13-196 sec
• generalized XORs ? 6.6 x 108 ? 9.2 x 1015
• Pure CSP does not scale limited by the number
  of search nodes that can be traversed within 1
  hour

(99 confidence)
21
Experimental Highlights, 2

• Graph coloring problems, e.g. games120
• Pure CSP ? 4.3 x 108 1 hr
• Pseudo-Boolean counter ? 1.1 x 106 0.5 hr
• Relsat (SAT counter) on solution ? 1.4 x 106 0.5
  hrcount preserving translation Morgado et
  al. 06, Bailleux et al. 06
• Generalized XORs ? 4.5 x 1042 1 min

22
Experimental Highlights, 3

• Spatially balanced Latin squares
• Impractical for SAT counters due to average
  distance computations
• Using streamlined version for experiments
• order 15 order 17
• pure CSP ? 112, 1 hr --- 1 hr
• generalized XORs ? 1748, 8 min ? 1058, 14
  min
• Note order 17 instance cannot be solved at all
  by pure CSP in 1 hr!

23
Summary

• 3 approaches to extend the XOR-based counting
  framework from SAT to CSPs
• Generalized XORs
• Advanced filtering / propagation techniques for
  XORs
• Experimental results very promising
• Quite fast in practice
• Very high counts
• Provable correctness guarantees