Towards a Searchable Space of Dynamical System Models

1
Towards a Searchable Space of Dynamical System
Models

• Eric Mjolsness
• Scientific Inference Systems Laboratory (SISL)
• University of California, Irvine
• www.ics.uci.edu/emj
• In collaboration with Guy Yosiphon
• NKS June 2006

2
Motivations shared with NKS

• Objective exploration of properties of simple
  computational systems
• Relation of such to the sciences
• Example bit string lexical ordering of cellular
  automata rules reducibility relationships
  applications to fluid flow

3
Criteria for a space of simple formal systems

• C1 Demonstrated expressive power in scientific
  modeling
• C2 Representation as discrete labeled graph
  structure
• that can be searched and explored computationally
• E.g. Bayes nets, Markov Random Fields
• roughly in order of increasing size - with index
  nodes (DDs)
• C3 Self-applicability
• useful transformations and searches of such
  dynamical systems should be expressible
• as discrete-time dynamical systems that compute
• So major changes of representation during
  learning are not excluded.

4
C1 Demonstration of expressive power in
scientific modeling
5
Elementary Processes

• A(x) ? B(y) C(z) with rf (x, y, z)
• B(y) C(z) ? A(x) with rr (y, z, x)
• Examples
• Chemical reaction networks w/o params
• .
• XXX from paper
• Effective conservation laws
• E.g. ? NA(x) dx ? NB(y) dy ,
• ? NA(x) dx ? NC(z) dz

6
Amino Acid Syntheses
Kmech Yang, et al. Bioinformatics 21 774-780,
2005 Amino acid synthesis Yang et al., J.
Biological Chemistry, 280(12)11224-32, , Mar 25
2005. GMWC modeling Najdi et al., J.
Bioinformatics and Comp. Biol., to appear 2006.
7
Example Anabaena Prusinkiewicz et al. model
G. Yosiphon, SISL, UCI
8
Example Galaxy Morphology
G. Yosiphon, SISL, UCI
9
Example Arabidopsis Shoot Apical Meristem (SAM)
10
Quantification of growth
Co-visualization of raw and extracted nuclei data
11
PIN1-GFP expression
Time-lapse imaging over 40 hrs (Marcus Heisler, Ca
ltech)
12
Dynamic Phyllotactic Model
Emergence of new extended, interacting objects
floral meristem primordia. DGs at 3 scales
- molecular - cellular - multicellular.
H. Jönnson, M. Heisler, B. Shapiro, E.
Meyerowitz, E. Mjolsness - Proc. Natl Acad. Sci.
1/06
13
Model simulation on growing template
14
Spatial Dynamics in Biological Development

• Reimplemented weak spring model in 1 page
• Applying to 1D stem cell niches with diffusion,
  in plant and animal tissues

15
Ecology predator-prey models
with Elaine Wong, UCI
16
Example Hierarchical Clustering
17
ML example Hierarchical Clustering
18
Logic Programming

• E.g. Horn clauses
• Rules
• Operators
• Project to fixed-point semantics

19
An Operator Algebra for Processes

• Composition is by independent parallelism
• Create elementary processes from yet more
  elementary Basis operators
• Term creation/annihilation operators for each
  parm value,
• Obeying Heisenberg algebra
• ai, cj di j or
• Yet classical, not quantum, probabilities

20
Basic Operator Algebra Composition Operations
,

• G Syntax
• parallel rules
• Multiple terms
• on LHS, RHS

• Operator algebra
• H1 H2
• H1 H2
• (noncommutative)

• Informal meaning
• independent, parallel occurrence
• instantaneous, serial
• co-occurrence

21
Time Evolution Operators

• Master equation
• d p(t) / dt H p(t)
• where 1H 0, e.g.
• H P(H) H - 1 diag(1H )
• H time evolution operator
• can be infinite-dimensional
• Formal solution
• p(t) exp(t H) p(0)

22
Discrete-Time Semantics of Stochastic
Parameterized Grammars
This formulation can also be used as a
programming language, expressing algorithms.
23
Algorithm DerivationConceptual Map
Operator Space (high dim)
Time Ordered Product Expansion
?(c)
Trotter Product Formula
DG rules
(H, et H)
Heisenberg Picture
Eulers formula
CBH
stochastic program
(H, Hn/(1 Hn p))
Functional Operator Space
?(d)
24
C2 Representation as discrete labeled graph
structure that can be searched and explored
computationally
25
Basic Syntax for a Modeling Language Stochastic
Parameterized Grammars (SPGs)

• G set of rules
• Each rule has
• LHS ? RHS keyword expression
• Parameterized term instances within LHS and/or
  RHS
• LHS, RHS sets (of such terms) with Variables
• LHS matches subsets of parameterized term
  instances in the Pool
• Keyword clauses specify probability rate, as a
  product
• Keyword with
• Algebraic sublanguage for probability rate
  functions
• rates are independent of of other matches
  oblivious.
• Rule/object verb/noun reaction/reactant
  bipartite graphs
• with complex labels

26
Graph Meta-Grammar
27
Plenum SPG/DG implementation

• builds on Cellerator experience
• Shapiro et al., Bioinformatics 19(5)677-678
  2003
• computer algebra embedding provides
• probability rate language
• Symbolic transformations to executability
• includes mixed stochastic/continuous sims

28
SPG/DG Expressiveness Subsumes

• Logic programming (w. Horn clauses)
• LHS ? RHS all probability rates equal
• Hence, any simulation or inference algorithms can
  in principle be expressed as discrete-time SPGs
• Chemical reaction networks
• No parameters stoichiometry weighted labeled
  bipartite graph
• Context-free (stochastic) grammars
• No parameters 1 input term/rule
• Formally solvable with generating functions
• Stochastic (finite) Markov processes
• No parameters 1 input/rule, 1 output/rule
• Solvable with matrices (or queuing theory?)

29
SPG/DG Expressiveness Subsumes

• Bayes Nets
• Each variable x gets one rule
• Unevaluated-term, evaluated predecessors(y) ?
  evaluated-term(x)
• MCMC dynamics
• Inverse rule pairs satisfying detailed balance
• Each rule can itself have the power of a
  Boltzmann distribution
• Probabilistic Object Models
• Frameville, PRM,
• Petri Nets
• Graph grammars
• Hence, meta-grammars and grammar transformations
• DGs subsume ODEs, SDEs, PDEs, SPDEs
• Unification with SPGs too

30
C3 Self-applicability

• Arrow reversal
• Arrow reversal graph grammar exercise
• Machine learning by statistical inference
• e.g. hierarchical clustering (reported)
• ? Equilibrium reaction networks for MRFs
• Further possible applications

31
Template A-Life

• Concisely expressed in SPGs
• Steady state condition total influx into g
  total outflow from g

32
Applications to Dynamic Grammar Optimizationand
a Grammar Soup

• Map genones to grammars
• Map hazards to functionality tests
• Map reproduction to crossover or simulation

33
Conclusions

• Stochastic process operators as the semantics for
  a language
• A fundamental departure
• Specializes to all other dynamics
• Deterministic, discrete-time, DE, computational,
  
• Graph grammars allow meta-processing
• Operator algebra leads to novel algorithms
• Wide variety of examples at multiple scales
• Sciences
• Cell, developmental biology astronomy geology
• multiscale integrated models
• AI
• Pattern Recognition
• Machine learning
• Searchable space of simple dynamical system
  models including computations

34
For More Information

• www.ics.uci.edu/emj ? modeling frameworks