Computational Complexity of Some Enumeration Problems in Sparse Network Automata

1
Computational Complexity of Some Enumeration
Problems in Sparse Network Automata

• Predrag Tosic
• p-tosic_at_cs.uiuc.edu
• Department of Computer Science
• University of Illinois at Urbana-Champaign
• Understanding Complex Systems Symposium
• May 15 18, 2006

2
What Is This Talk All About

• Studying possible computations / dynamics of
• CA-like discrete dynamical systems
• Graph/Network Automata models of interest
• Sequential and Synchronous Dynamical Systems
• (SDS and SyDS), Discrete Hopfield Networks
• Configuration space properties of interest
• counting Fixed Points, Gardens of Eden,
• predecessor states, other types of
  configurations
• Main results counting FPs is P-complete, even
  when the SDSs and SyDSs are severely restricted

3
Summary of Results

• Counting Fixed Points (FPs) of SDSs and SyDSs
  exactly or approximately is intractable even if
• the update rules are symmetric Boolean
  functions
• the update rules are monotone linear
  threshold functions
• the update rules are symmetric, monotone
  linear threshold
• Boolean functions (simple threshold)
• Intractability of counting holds even when
• - only two different rules from the class are
  used, and
• - each node has only O (1) neighbors
• (In contrast) counting FPs in e.g. 1-D Simple
  Threshold CA is tractable

4
Talk Outline

• Introduction and Motivation
• - Why counting matters some motivating
  examples
• Preliminaries
• - Introducing SDS and SyDS models
• - Computational complexity of enumeration
  problems
• Complexity of Counting FPs
• Related Work
• Summary and Some Open Problems

5
Importance of Counting in Complex Dynamical
Systems

• Reliability and fault-tolerance in networks
• Discrete Hopfield Networks
• - Model of associative memory
• - FP problem how many patterns can be
  stored?
• Statistical physics
• - Ising model and spin glasses
• computing the partition function
• - determining degeneracy in energy spectra

6
Network Connectedness Fault-Tolerance Counting
Paths
7
Statistical Physics Spin Glasses
8
Counting in Dynamical Systems

• Many important applications
• - determining the size of the basin of
  attraction of an attractor
• - how many patterns can be stored in an
  associative memory
• Very few theoretical results
• - proving hardness or easiness of counting is
  difficult
• - determining complexity of counting in
  sparse (but static )
• combinatorial structures is challenging
• - complexity of counting in dynamical
  systems even more so
• Our results are a breakthrough, esp. in the
  context of linear threshold simple threshold
  update rules

9
Cellular Automata (CA)

• Network or regular grid of very simple
  processors at individual nodes
• In classical CA each processor is a
  deterministic FSM
• Definition CA A (G, N, M) where
• - G is a cellular space
• regular graph set of node states
• - N is a fundamental neighborhood
• - M is a (deterministic) Finite State
  Machine

10
Some Examples of Dynamics of 1D CA
11
Some Variants of Cellular Automata

• Cellular space
• - in classical CA, any regular graph
• - in graph automata (GA), an arbitrary
  (finite) graph
• Node update rule
• - a simple function specifying how future
  state of a node
• depends on its current state and current
  states of this
• nodes pre-specified neighbors
• Node update ordering
• - classical CA all in parallel (perfect
  synchrony)
• - sequential node updates one node at a time

12
Some Related Models of Complex Systems

• Classical parallel sequential CA
• Connectionist AI Hopfield Networks
• Theoretical biology S. Kauffmans Random Boolean
  Networks
• Graph Automata models of B. Martin, Nichitiu
  Remila, S. Roka

13
Various Network Automata Models

• One-way CA (e.g., One-Way CA on Cayley Graphs
  model by S. Roka, in TCS vol. 132 (1-2), 94
  )
• Graph Automata of B. Martin and C. Nichitiu, E.
  Remila
• (both in Proc. MFCS 98 the latter
  Fin-Dom-SyDS )
• Sequential node updates e.g., R. Laubenbacher,
  B. Pareigis, in TR, NMSU, Las Cruces, 00
  also B. Huberman, N. Glance on dynamics in games,
  in Proc. Natl Acad. Sci., 99
• Foundational work on SDS in the context of
  developing theory of large-scale computer
  simulations (TRANSIMS )
• - C. Barrett et al. various papers, 1999
  2003
• - C. Reidys, H. Mortveit Appl. Math.
  Comput. 00, 01

14
Sequential Synchronous Dynamical Systems (SDS
SyDS)

• Cellular space an arbitrary finite graph
• Heterogeneous local behaviors
• different nodes update according to different
  rules
• In SDS nodes update sequentially one at a
  time, according to a fixed permutation
• In SyDS nodes update in parallel, perfectly
  synchronously ( just like in classical CA )

15
Definition of SDS SyDS

• SDS S is a triple (G, F, ?) where
• - G G(V, E) is the underlying graph
• - F (f1, , fn) is the global map
• - ? is a fixed permutation of the nodes (x1,
  , xn)
• ? specifies sequential ordering of node updates
• fi DNi 1 ! D is the i-th nodes
  update rule
• SyDS S is like an SDS, only without the node
  permutation (synchronous parallel updates)

16
An Example of a Simple Threshold SDS
17
Computational Complexity of Enumeration Problems

• The problems of computing the permanent and
  counting perfect matchings in bipartite graphs
• (L. Valiant in SIAM J. Comp.79, TCS 79)
• Class P those counting problems accepted by
  poly-time bounded NTM s.t. the of accepting
  computations equals the of problems
  solutions
• P-complete problems the hardest in P
• NP-complete decision problems have their counting
  versions in P but so do several problems in P
• (e.g., bipartite matchings, 2CNF SAT,
  MON-2CNF SAT)

18
PH, P and PSPACE
EXP
PSPACE

• PSPACE
• Complete problems TQBF, 2-person games,
  generalized
• geography, REACHABILITY for finite CA, SDS
• 3rd level
• Complete problems VC dimension for succinct sets
• Containment Approximate P
• 2nd level
• Complete problems Min DNF, Succinct Set Cover
• Containment Min Circuit, SGEE for SDS, class
  BPP
• 1st level
• Complete problems SAT, UNSAT, 3CNF-SAT,
• FPE, GEE, GE, SGE for SDS
• Containment factoring, graph isomorphism,

P
PH ...
S3P
?3P
?3P
S2P
?2P
?2P
NP
coNP
BPP
P
19
How to Prove P-Completeness

• We need efficient (poly-time) reductions that
• preserve the of solutions
• Parsimonious Reductions Polynomial time
  transformations that exactly preserve the of
  solutions ?(f(I)) ?(I)
• Weakly Parsimonious Reductions Poly-time
• transformations f that allow the of
  solutions of
• ?(I) to be efficiently recovered from
  ?(f(I))

20
Results That Count

• Computational complexity of counting various
  types of configurations in S(y)DS
• - Fixed Points (stable states)
• - Gardens of Eden (unreachable states)
• - Predecessors of a given configuration
• Hardness holds even as we severely restrict
• - the allowed local rules fi
• - of different rules used
• - underlying graphs G

21
Counting Complexity in SDS / SyDS General
Strategy

• Constructing S(y)DSs from restricted classes of
  Boolean CNF formulae (e.g., 3CNF, Mon-2CNF)
• Proving reductions are (weakly) parsimonious
• Tightening the rope by restricting
• - the class of allowed node update rules
• - how many different rules are allowed to
  be used
• - the structure of underlying graphs
• (cf. in terms of the bounds on of
  neighbors)

22
Symmetric S(y)DS

• Modeling mean-field effects in statistical
  physics, engineering applications
• Each update rule fi depends only on how many
• nodes are currently in state 1 (i.e., only on
  their sum)
• Optimally succinct truth tables are of linear
  
• (instead of exponential ) size
• Applying to more general cellular spaces
• Graph/Network Automata with symmetric update
  rules
• Previous work on Boolean Symmetric S(y)DS
• (Barrett et al. in DMTCS01, TCS03)

23
Counting in Sym-Bool SDS with Bounded Node Degrees

• Theorem 1 ECCC05-TR051
• (i) Counting FPs, TCs exactly in Sym-Bool
  
• S(y)DSs with all node degrees O (1)
• is P-complete
• (ii) Counting FPs approximately to within
• 2n1 - ? in Sym-Bool S(y)DSs with
• O (1)-bounded node degrees is NP-hard

24
Pushing the Node Degree Limit

• Theorem 2
• Counting FP of Sym-Bool-S(y)DSs
• is P-complete even if
• - all the nodes are of degree at most /
  exactly
• equal to 3
• - the underlying graph is bipartite
• For details, see P.T., G. Agha in Proc. EUMAS05

25
Linear Threshold Functions

• Boolean-valued linear threshold functions
  
• xi à 1, if ? wij xj ?
• 0, otherwise
• If all weights wij 0, then the function
  is monotone
• If all weights are positive and equal
  (w.l.o.g., wij 1),
• then the update rule is both monotone
  symmetric
• Those are called simple threshold functions
• Some examples MAJORITY, AND, OR

26
Complexity of Monotone Boolean Formulae/Circuits/A
utomata

• Decision problems tend to be easy
• FP existence for symmetric S(y)DSs is
  NP-complete DMTCS01, but trivial for monotone
  S(y)DSs
• Counting problems on monotone structures
  however are often hard (e.g., MON-2CNF
  Satisfiability )
• We prove hardness of counting FPs in monotone
  SDSs and SyDSs restricted to linear threshold
  rules
• Hardness holds for
• - planar, bipartite, sparse on average
  S(y)DSs IJFCS06
• - bipartite, uniformly sparse S(y)DSs (this
  talk)

27
Counting FPs of Monotone Linear Threshold S(y)DS

• Theorem 3 Counting FP of Monotone Boolean
  SDSs and SyDSs is P-complete even when all of
  the following conditions simultaneously hold
• - the monotone update rules are linear
  threshold
• functions with all wij 0
• - S(y)DSs are with memory, and s. t. wii
  1
• - only two different integer weights are used
• - each node has at most 3 neighbors

28
Proof Outline for Theorem 3

• Reduction is from MON-2CNF
• Each clause node Cj uses the linear threshold
  function
• Cj à Bool(2 Cj Cj xj1 xj2 4)
• Variable nodes xi and cloned clause nodes Cj
  compute Boolean AND ( which can also be
  expressed as a linear threshold function)
• Total of two positive integer weights are used

29
Construction in Proof of Theorem 3
30
Complexity of Counting FPs in Simple Threshold
SDSs SyDSs

• Theorem 4
• (i) Determining FP (also GE, TC) exactly in
  Simple
• Threshold ( monotone symmetric ) SDS
  and
• SyDS is P-complete
• (ii) Determining FP approximately is
  NP-hard
• The sharpest result yet
• - update rules are symmetric, monotone and
  linear threshold
• - using only two distinct fi still suffices
• - (i) for FPs holds when node degrees are
  bounded by / equal to 4

31
Related Work on Configuration Space Properties of
S(y)DS

• Mathematical characterization of GE, FP, TC
• (C. Barrett et al., Appl. Math. Comput.
  99 00 01)
• Computational Complexity of Fixed Point,
• Garden of Eden existence (Barrett et al.,
  DMTCS 01)
• Inverse dynamics (Barrett et al. in LANL TR
  01)
• Reachability problems
• (e.g., Barrett et al., Annals Comb. 02 TCS
  03)

32
Related Results on the Complexity of Counting

• (Perfect) Matchings in Bipartite Graphs
• L. Valiant, SIAM J. Comp. 79
• - perfect match. remains hard for k-regular
  bipartite graphs, 8 k 3
• - all matchings remains hard for
  bipartite graphs with node degrees 4,
• bipartite planar with node deg. 6
  S. Vadhan, SIAM J. Comp. 01
• Planar 3CNF SAT H. Hunt et al., TCS 98
• Bipartite MON-2CNF SAT
• - planar graph, appearances 4 S.
  Vadhan, SIAM J. Comp. 01
• MON-2CNF SAT
• - appearances k, 8 k 5 S. Vadhan, SIAM
  J. Comp. 01
• - P-complete for k 3, 4 C. Greenhill,
  TCS 00
• Horn 2CNF SAT, appearances 4 D. Roth, AI
  96

33
Hopfield Networks
34
Counting in Hopfield Networks

• FPs how many solutions to
• x sgn(W x) such that x 2 -1, 1n
• where W wij is the weight matrix ?
• PREDECESORS how many y 2 -1, 1n solve the
  inverse problem
• x sgn(W y) for given x 2 -1, 1n
• ANCESTORS how many patterns eventually
  converge to a given pattern (of minimal energy) ?

35
Counting FPs in Other Discrete Dynamical Systems
with Linear Threshold Rules

• Classical work by Floreen and Orponen on Discrete
  Hopfield Nets (DHNs)
• Counting FPs in DHNs with linear threshold rules
  
• is hard (Floreen Orponen, Complex
  Systems, 1989)
• However
• - they use both positive and negative weights
• - many different weights wij are used
• - either wii 0 or else some negative wii
• - some nodes have ?(n) neighbors
• We considerably simplify the weight matrix

36
Counting FPs in Sparse Discrete Hopfield Networks

• Corollary to Theorem 3
• Exactly enumerating FPs of a Discrete Hopfield
  Network is P-complete even when the following
  simultaneously hold
• - nonnegative integer weight matrices
• - symmetric weight matrices with wii 1
• - only two nonzero weights used
• - exactly / at most four wij ? 0 per row

37
Counting in Simple Threshold S(y)DS vs. 1-D (S)CA

• Focus on of local update rules used
• An S(y)DS can use only two simple threshold
  rules, and FP problem remains hard
• What if all nodes use the same rule ?
• (Partial) Answer We consider 1-D Sequential and
  Parallel CA, and show that determining FP in
  those CA is tractable in principle
• 1-D CA with the MAJORITY rule
• Combinatorics is involved, but FP 2 P

38
Fixed Points of 1-D MAJ (S)CA

• Theorem 5
• For any integer r 1, 1-D MAJ (S)CA with
  n nodes
• and of radius r have exponentially many
  FPs
• FP can be efficiently computed
• MAJ (S)CA on infinite cellular spaces have
  
• uncountably many FPs
• Corollary
• Only a handful of FPs if ? ? MAJ
• FP can be effectively determined for ?
  MAJ

39
Summary

• Theme computational complexity of counting
  different configurations (cf. FPs) in CA-like
  Hopfield Net-like discrete dynamical systems
• Counting FPs in SDS and SyDS is P-complete even
  when all of the following conditions
  simultaneously hold
• - symmetric / totalistic update rules
• - monotone linear threshold update rules
• - only two different update rules are used
• - the underlying graph is k-regular, for
  k 4
• Approximately counting FPs in these restricted
  classes of
• SDSs and SyDSs is NP-hard in general

40
Some Open Problems

• Is counting FPs in Simple Threshold SDSs s.t.
  each node has at most / exactly 3 neighbors still
  hard ?
• Is counting in Simple Threshold CA still easy
  for more general, higher-dimensional cellular
  spaces ?
• Can counting FPs or other types of configurations
  still be hard if all the nodes behave the same
  ?
• (partial answer counting PREDECESSORS can)
• Finding important restricted versions of SDSs
  (if there are any) where counting is
  computationally tractable
• Further implications for other discrete dynamical
  system models (e.g., multi-layer ANNs, discrete
  HNs)

41
References

• P.T. On Complexity of Counting Fixed Point
  Configurations in Certain Classes of Cellular and
  Graph Automata, ECCC-TR05-051, April 2005
• P.T. Counting Fixed Points and Gardens of Eden
  of Sequential Dynamical Systems on Planar
  Bipartite Graphs, ECCC-TR05-091, August 2005
• P.T., G. Agha. On Complexity of Counting Fixed
  Points in Symmetric Boolean Graph Automata,
  Proceedings of the Fourth International
  Conference on Unconventional Computation (UC05),
  Springer LNCS 3699, October 2005
• P.T., G. Agha. On the Computational Complexity
  of Predicting Dynamical Evolution of Large Agent
  Ensembles, Proceedings of the Third European
  Workshop on Multi-Agent Systems (EUMAS), December
  2005
• P.T. On the Complexity of Counting Fixed Points
  and Gardens of Eden in Sequential and Synchronous
  Dynamical Systems on Planar Bipartite Graphs,
  accepted to International Journal on Foundations
  of Computer Science, January 2006
• P.T. Computational Complexity of Counting Fixed
  Points and Other Configurations in Sparse
  Cellular and Graph Automata, full journal
  version, to be submitted