Graphical Models - Inference -

1
Graphical Models- Inference -
Mainly based on F. V. Jensen, Bayesian Networks
and Decision Graphs, Springer-Verlag New York,
2001.
Advanced I WS 06/07
Variable Elimination

• Wolfram Burgard, Luc De Raedt, Kristian
  Kersting, Bernhard Nebel

Albert-Ludwigs University Freiburg, Germany
2
Outline

• Introduction
• Reminder Probability theory
• Basics of Bayesian Networks
• Modeling Bayesian networks
• Inference
• Excourse Markov Networks
• Learning Bayesian networks
• Relational Models

3
Elimination in Chains

• Forward pass
• Using definition of probability, we have

- Inference (Variable Elimination)
4
Elimination in Chains

• By chain decomposition, we get

- Inference (Variable Elimination)
5
Elimination in Chains

• Rearranging terms ...

- Inference (Variable Elimination)
6
Elimination in Chains

• Now we can perform innermost summation

- Inference (Variable Elimination)
7
Elimination in Chains
X

• Rearranging and then summing again, we get

- Inference (Variable Elimination)

8
Elimination in Chains with Evidence

• Similar due to Bayes Rule
• We write the query in explicit form

- Inference (Variable Elimination)
9
Variable Elimination

• Write query in the form
• Iteratively
• Move all irrelevant terms outside of innermost
  sum
• Perform innermost sum, getting a new term
• Insert the new term into the product

Semantic Bayesian Network
Eliminate irrelevant variables
- Inference (Variable Elimination)
10
Asia - A More Complex Example
- Inference (Variable Elimination)
11
We want to compute P(d)Need to eliminate
v,s,x,t,l,a,b

• Initial factors

- Inference (Variable Elimination)
12
We want to compute P(d)Need to eliminate
v,s,x,t,l,a,b

• Initial factors

Eliminate v
- Inference (Variable Elimination)
13
We want to compute P(d)Need to eliminate
s,x,t,l,a,b

• Initial factors

Eliminate s
- Inference (Variable Elimination)
Summing on s results in a factor with two
arguments fs(b,l). In general, result of
elimination may be a function of several variables
14
We want to compute P(d)Need to eliminate
x,t,l,a,b

• Initial factors

- Inference (Variable Elimination)
Eliminate x
Note that fx(a)1 for all values of a
15
We want to compute P(d)Need to eliminate t,l,a,b

• Initial factors

- Inference (Variable Elimination)
Eliminate t
16
We want to compute P(d)Need to eliminate l,a,b

• Initial factors

- Inference (Variable Elimination)
Eliminate l
17
We want to compute P(d)Need to eliminate a,b

• Initial factors

- Inference (Variable Elimination)
Eliminate a,b
18
Variable Elimination (VE)

• We now understand VE as a sequence of rewriting
  operations
• Actual computation is done in elimination step
• Exactly the same procedure applies to Markov
  networks
• Computation depends on order of elimination

- Inference (Variable Elimination)
19
Dealing with Evidence

• How do we deal with evidence?
• Suppose get evidence V t, S f, D t
• We want to compute P(L, V t, S f, D t)

- Inference (Variable Elimination)
20
Dealing with Evidence

• We start by writing the factors

- Inference (Variable Elimination)
21
Dealing with Evidence

• We start by writing the factors
• Since we know that V t, we dont need to
  eliminate V
• Instead, we can replace the factors P(V) and
  P(TV) with

- Inference (Variable Elimination)
22
Dealing with Evidence

• We start by writing the factors
• Since we know that V t, we dont need to
  eliminate V
• Instead, we can replace the factors P(V) and
  P(TV) with
• This selects the appropriate parts of the
  original factors given the evidence
• Note that fP(v) is a constant, and thus does not
  appear in elimination of other variables

- Inference (Variable Elimination)
23
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence

- Inference (Variable Elimination)
24
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence
• Eliminating x, we get

- Inference (Variable Elimination)
25
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence
• Eliminating x, we get
• Eliminating t, we get

- Inference (Variable Elimination)
26
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence
• Eliminating x, we get
• Eliminating t, we get
• Eliminating a, we get

- Inference (Variable Elimination)
27
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence
• Eliminating x, t, a, we get
• Finally, when eliminating b, we get

- Inference (Variable Elimination)
28
Dealing with Evidence
Given evidence V t, S f, D t, compute P(L,
V t, S f, D t )

• Initial factors, after setting evidence
• Eliminating x, t, a, we get
• Finally, when eliminating b, we get

- Inference (Variable Elimination)
29
Outline

• Introduction
• Reminder Probability theory
• Basics of Bayesian Networks
• Modeling Bayesian networks
• Inference (VE, Junction Tree)
• Excourse Markov Networks
• Learning Bayesian networks
• Relational Models