MPE and Partial Inversion in Lifted Probabilistic Variable Elimination

1
MPE and Partial Inversion inLifted Probabilistic
Variable Elimination

• Rodrigo de Salvo Braz
• University of Illinois at
• Urbana-Champaign

with Eyal Amir and Dan Roth
2
Lifted Probabilistic Inference

• We assume probabilistic statements such as8
  Person, DiseaseP(sick(Person,Disease)
  epidemics(Disease)) 0.3
• Typical approach is grounding.
• We seek to do inference at first-order level,
  like it is done in logic.
• Faster and more intelligible.
• Two contributions
• Partial inversion more general technique than
  previous work (IJCAI '05)
• MPE and Lifted assignments

3
Representing structure
epidemic(measles)
epidemic(flu)



sick(mary,measles)
sick(mary,flu)
sick(bob,measles)
sick(bob,flu)



Poole (2003) named these parfactors, for
parameterized factors
Logical variable
epidemic(D)
Atom
sick(P,D)
4
Parfactor
8 Person, Disease f(sick(Person,Disease),
epidemic(Disease))
5
Parfactor
Person ? mary, Disease ? flu
8 Person, Disease f(sick(Person,Disease),
epidemic(Disease)), Person ? mary, Disease ? flu
6
Joint Distribution

• As in propositional case, proportional to product
  of all factors
• But here, all factors means all instantiations
  of all parfactors
• P(...) ? ÕX f1(p(X)) ÕX,Y f2(p(X),q(X,Y))

7
Inference task - Marginalization

• åq(X,Y) ÕX f1(p(X)) ÕX,Y f2(p(X),q(X,Y))
• Marginal on all random variables in p(X)
• summation over all assignments to all instances
  of q(X,Y)

8
The FOVE Algorithm

• First-Order Variable Elimination (FOVE) a
  generalization of Variable Elimination in
  propositional graphical models.
• Eliminates classes of random variables at once.

9
FOVE

• P(hospital(mary)) ?

epidemic(measles)
epidemic(D)
D ? measles
sick(mary,measles)
sick(mary, D)
D ? measles
hospital(mary)
10
FOVE

• P(hospital(mary)) ?

epidemic(D)
D ? measles
sick(mary,measles)
sick(mary, D)
D ? measles
hospital(mary)
11
FOVE

• P(hospital(mary)) ?

epidemic(D)
D ? measles
sick(mary, D)
D ? measles
hospital(mary)
12
FOVE

• P(hospital(mary)) ?

D ? measles
sick(mary, D)
D ? measles
hospital(mary)
13
FOVE

• P(hospital(mary)) ?

hospital(mary)
14
Counting Elimination - A Combinatorial Approach

• åe(D) ÕD1?D2 f(e(D1),e(D2))
• åe(D) f(0,0)(0,0) in assignment
  f(0,1)(0,1) in assignment
  f(1,0)(1,0) in assignment
  f(1,1)(1,1) in assignment
• Let i be the number of e(D)s assigned 1
• å i Õv1,v2 f(v1,v2)(v1,v2) given i
• ? (number of assignments with D
  e(D)1 i)

15
Counting Elimination - Conditions

• It does not work oneliminating class epidemic
  fromf(epidemic(D1, Region), epidemic(D2,
  Region), donations).
• In general, counting elimination does not apply
  when atoms share logical variables.
• Here, Region is shared between atoms.

16
Partial Inversion
Provides a way of not sharing logical
variables åe(D,R) ÕD1?D2,R f( e(D1,R), e(D2,R), d
) ÕR åe(D,r) ÕD1?D2 f( e(D1,r), e(D2,r), d ) (R
is now bound, so not a variable anymore) ÕR f(
d ) f( d )R f( d )
17
Partial Inversion, graphically
epidemic(D1,R)
Each instance a counting elimination problem
donations
D1 ? D2
epidemic(D2,R)

epidemic(D1,r1)
epidemic(D1,r10)
D1 ? D2

D1 ? D2
epidemic(D2,r1)
epidemic(D2,r10)
donations
18
Another (not so partial) inversion

• åq(X,Y) ÕX,Y f(p(X),q(X,Y)) (expensive)
• ÕX,Y åq(X,Y) f(p(X),q(X,Y)) (propositional)
• ÕX,Y f'(p(X))
• ÕX f'Y(p(X))
• ÕX f''(p(X)) (marginal on p(X))

19
Another (not so partial) inversion
p(X)
Each instance a propositional elimination problem
q(X,Y)

p(x1)
p(xn)

q(x1,y1)
q(xn,yn)
20
Partial inversion conditions
f( friends(X,Y), friends(Y,X)) Cannot partially
invert on X,Y because friends(bob,mary) appears
in more than one instance of parfactor.
21
Summary of Partial Inversion

• More general than previousInversion Elimination.
• Generates Counting Elimination or Propositional
  sub-problems.
• Cannot be applied to entangled parfactors.
• Does not depend on domain size.

22
Second contribution Lifted MPE

• In propositional case,MPE done by factors
  containing MPE of eliminated variables.

C
A
B
D
23
MPE

• In propositional case,MPE done by factors
  containing MPE of eliminated variables.

B D f MPE
0 0 0.3 C1
0 1 0.2 C1
1 0 0.5 C0
1 1 0.9 C1
A
B
D
24
MPE

• In propositional case,MPE done by factors
  containing MPE of eliminated variables.

B f MPE
0 0.5 C1,D0
1 1.4 C1,D1
A
B
25
MPE

• In propositional case,MPE done by factors
  containing MPE of eliminated variables.

A f MPE(B,C,D)
0 0.9 B0,C1,D0
1 0.7 B1,C1,D1
A
26
MPE

• In propositional case,MPE done by factors
  containing MPE of eliminated variables.

f MPE
0.9 A0,B1,C1,D1
27
MPE

• Same idea in First-order case
• But factors are quantified and so are
  assignments

8 X, Y f(p(X), q(X,Y))
p(X) q(X,Y) f MPE
0 0 0.3 r(X,Y) 1
0 1 0.2 r(X,Y) 1
1 0 0.5 r(X,Y) 0
1 1 0.9 r(X,Y) 1
28
MPE
8 X, Y f(p(X), q(X,Y))
p(X) q(X,Y) f MPE
0 0 0.3 r(X,Y) 1
0 1 0.9 r(X,Y) 1
1 0 0.5 r(X,Y) 0
1 1 0.3 r(X,Y) 1
Liftedassignments

• After Inversion Elimination of q(X,Y)

8 X f(p(X))
p(X) f MPE
0 0.05 8 Y q(X,Y) 1, r(X,Y) 1
1 0.02 8 Y q(X,Y) 0, r(X,Y) 1
29
MPE
8 X f(p(X))
p(X) f MPE
0 0.05 8 Y q(X,Y) 1, r(X,Y) 1
1 0.02 8 Y q(X,Y) 0, r(X,Y) 1

• After Inversion Elimination of p(X)

f()
f MPE
0.009 8 X 8 Y p(X) 0, q(X,Y) 1, r(X,Y) 0
30
MPE
8 D1, D2 f(e(D1), e(D2))
e(D1) e(D2) f MPE
0 0 0.3 r(D1,D2) 1
0 1 0.9 r(D1,D2) 1
1 0 0.5 r(D1,D2) 0
1 1 0.3 r(D1,D2) 1
f()

• After Counting Elimination of e

f MPE
0.05 938 D1,D2 e(D1)0, e(D2) 0, r(D1,D2) 1912 D1,D2 e(D1)0, e(D2) 1, r(D1,D2) 1915 D1,D2 e(D1)1, e(D2) 0, r(D1,D2) 0925 D1,D2 e(D1)1, e(D2) 1, r(D1,D2) 1
31
Conclusions

• Partial InversionMore general algorithm,
  subsumes Inversion elimination
• Lifted Most Probable Explanation (MPE)
• same idea as in propositional VE, but with
• Lifted assignments
• describe sets of basic assignments
• universally quantified comes from Partial
  Inversion
• existentially quantified comes from Counting
  elimination
• Ultimate goal
• to perform lifted probabilistic inference in way
  similar to logic inference without grounding and
  at a higher level.

32
(No Transcript)