Independence, Decomposability and functions which take values into an Abelian Group

1
Independence, Decomposability and functions which
take values into an Abelian Group

• Adrian Silvescu
• Vasant Honavar

Department of Computer Science Iowa State
University
2
Decomposition and Independence

• Decomposition renders problems more tractable.
• Apply recursively
• Decomposition is enabled by independence
• Decomposition and independence are dual notions

A B
A
B
A
B
A
B
3
Conditional Decomposition and Independence

• Seldom are the two sub-problems disjoint
• All is not lost
• Conditional Decomposition / Independence
• Conditioning on C
• C a.k.a. separator

C
A
B
C
A
B
A
C
C
B
C
A
B
4
Formalization of the intuitions

• Problem P (D, S, solP)
• D Domain, S Solutions
• solP D S

solP
A
B
Example Determinant_Computation(M2, R, det)
5
Conditional Independence / Decomposition
Formalization (Variable Based)

• P (D A X B X C, S, solP)
• P1 (A X C, S1, solP1), P2 (B X C, S2,
  solP2)
• solP(A, B, C) solP1(A, C)
  solP2(B, C)

6
Probabilities

• I(A, BC) iff P(A, B C) P(AC) P(BC)
• Equivalently P(A, B, C) P(A, C) P(BC)
• P(A, B, C) f1(A, C) f2(B, C)
• Independencies can be represented by a graph
  where we do not draw edges between variables that
  are independent conditioned on the rest of the
  variables.

7
The Hammersley-Clifford Theorem From Pairwise
to Holistic Decomposability
8
Outline

• Generalized Conditional Independence with
  respect to a function f and properties
• Theorems
• Conclusions and Discussion

9
Conditional Independence with respect to a
function f - If(A,BC)

• solP(A, B, C) solP1(A, C)
  solP2(B, C)
• Assumptions
• S S1 S2 G
• 
  .
• A, B, C is a partition of the set of all
  variables
• Saturated independence statements from now on
  
  

• f(A, B, C) f1(A, C) f2(B, C)

If(A,BC)
10
Conditional Independence with respect to a
function f If(A,BC) contd

• If(A,BC) iff
• f(A, B, C) f1(A, C) f2(B, C)

11
Examples of If(A,BC )

• Multiplicative (probabilities)
• Additive (fitness, energy, value functions)
• Relational (relations)

12
Properties of If(A,BC )

• 1.Trivial Independence
• If(A, FC)
• 2. Symmetry
• If(A, BC) gt If(B, AC)
• 3. Weak Union
• If(A, B U DC) gt If(A, BC U D)
• 4. Intersection
• If(A, BC U D) If(A, DC U B) gt If(A, B U
  DC)

A
C
D
B
13
Abelian Groups

• (G, , 0, -) is an Abelian Group iff
• is associative and commutative
• 0 is a neutral element
• - is an inversion operator
• Examples
• (R, , 0, - ) - additive (value func.)
• ((0, 8), , 1, -¹) - multiplicative (prob.)
• (0, 1, mod2, 0, id) - relational (relations)

14
Outline

• Generalized Conditional Independence with
  respect to a function f
• Properties and Theorems
• Conclusions and Discussion

15
Markov Properties Pearl Paz 87

• If Axioms 1-4 then the following are equivalent
• Pairwise (a,ß) G gt If(a, ßV\a,ß)
• Local -
• If(a, V\(N(a)Ua) N(a))
• Global If CV\A, B separates A and B in G
  If(A, B CV\A, B)

16
Factorization Main Theorem
17
The Factorization Theorem From Pairwise to
Holistic Decomposability
18
Particular Cases - Factorization

• Probabilistic Hammersley-Clifford
• Additive Decomposability
• Relational Decomposability

19
Graph Separability and Independence Geiger
Pearl 93
If Axioms 1-4 hold then
SepG(A, BC) ? If(A, BC) for all saturated
independence statements
20
Completeness

• Axioms 1-4 provide a complete axiomatic
  characterization of independence statements for
  functions which take values over Abelian groups

21
Outline

• Generalized Conditional Independence with
  respect to a function f
• Properties and Theorems
• Conclusions and Discussion

22
Conclusions (1)

• Introduced a very general notion of Conditional
  Independence / Decomposability.
• Developed it into a notion of Conditional
  Independence relative to a function f which takes
  values into an Abelian Group If(.,..).
• We proved that If(.,..) satisfies the following
  important independence properties
• 1. Trivial independence,
• 2. Symmetry,
• 3. Weak union
• 4. Intersection

23
Conclusions (2)

• Axioms 1-4 imply the equivalence of the Global,
  Local and Pairwise Markov Properties for our
  notion conditional independence relation
  If(.,..)) based on the result from Pearl and
  Paz '87.
• We proved a natural generalization of the
  Hammersley-Clifford which allows us to factorize
  the function f over the cliques of an associated
  Markov Network which reflects the Conditional
  Independencies of subsets of variables with
  respect to f.
• Completeness Theorem, Graph Separability Eq.
  Theorem
• The theory developed in this paper subsumes
  probability distributions, additive decomposable
  functions and relations, as particular cases of
  functions over Abelian Groups.

24
Discussion Relation to Graphoids

• (-) Decomposition
• (-) Contraction
• () Weak Contraction
• Graphoids No finite axiomatic charact. Studeny
  92
• Intersection Discussion noninvertible elms.

25
Discussion contd

• Graph Separability ? Independence
• Completeness
• Seems that
• Trivial Independence
• Symmetry
• Weak Union
• Intersection
• Strong Axiomatic core for Independence

26
Applications