# Bayesian Networks Bucket Elimination Algorithm - PowerPoint PPT Presentation

PPT – Bayesian Networks Bucket Elimination Algorithm PowerPoint presentation | free to download - id: 681bff-OTQ1M

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Bayesian Networks Bucket Elimination Algorithm

Description:

### Title: Bayesian Networks Variable Elimination Algorithm Author: Tai-Wen Yue Last modified by: Tai-Wen Yue Created Date: 8/13/2002 6:49:27 AM Document presentation format – PowerPoint PPT presentation

Number of Views:6
Avg rating:3.0/5.0
Slides: 58
Provided by: TaiWe3
Category:
Tags:
Transcript and Presenter's Notes

Title: Bayesian Networks Bucket Elimination Algorithm

1
Bayesian Networks Bucket Elimination Algorithm
• ??????
• ???????
• ?????????

2
Content
• Basic Concept
• Belief Updating
• Most Probable Explanation (MPE)
• Maximum A Posteriori (MAP)

3
Bayesian Networks Bucket Elimination Algorithm
• Basic Concept
• ???????
• ?????????

4
Satisfiability
Given a statement of clauses (in disjunction
normal form), the satisfiability problem is to
determine whether there exists a truth assignment
to make the statement true.
Examples
Satisfiable
ATrue, BTrue, CFalse, DFalse
Satisfiable?
5
Resolution
can be true if and only if
can be true.
?
?
unsatisfiable
6
Direct Resolution
Example
Given a set of clauses
and an order dABCD
Set initial buckets as follows
7
Direct Resolution
Because no empty clause (???) is resulted, the
statement is satisfiable.
How to get a truth assignment?
8
Direct Resolution
9
Direct Resolution
10
Queries on Bayesian Networks
• Belief updating
• Finding the most probable explanation (mpe)
• Given evidence, finding a maximum probability
assignment to the rest of variables.
• Maximizing a posteriori hypothesis (map)
• Given evidence, finding an assignment to a subset
of hypothesis variables that maximize their
probability.
• Maximizing the expected utility of the problem
(meu)
• Given evidence and utility function, finding a
subset of decision variables that maximize the
expected utility.

11
Bucket Elimination
• The algorithm will be used as a framework for
various probabilistic inferences on Bayesian
Networks.

12
Preliminary Elimination Functions
Given a function h defined over subset of
variables S, where X ? S,
Eliminate parameter X from h
Defined over U S X.
13
Preliminary Elimination Functions
Given a function h defined over subset of
variables S, where X ? S,
14
Preliminary Elimination Functions
Given function h1,, hn defined over subset of
variables S1,, Sn, respectively,
Defined over
15
Preliminary Elimination Functions
Given function h1,, hn defined over subset of
variables S1,, Sn, respectively,
16
Bayesian Networks Bucket Elimination Algorithm
• Belief Updating
• ???????
• ?????????

17
Goal
Normalization Factor
18
Basic Concept of Variable Elimination
Example
19
Basic Concept of Variable Elimination
Example
20
Basic Concept of Variable Elimination
?G(f)
?D(a, b)
?F(b, c)
?B(a, c)
?C(a)
21
Basic Concept of Variable Elimination
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
22
Basic Concept of Variable Elimination
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
23
Basic Concept of Variable Elimination
f ?G(f )
0.1
? 0.7
24
Basic Concept of Variable Elimination
f ?G(f )
0.1
? 0.7
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
25
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
f ?G(f )
0.1
? 0.7
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
26
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
f ?G(f )
0.1
? 0.7
a c ?B(a, c)
0 0 0.9?0.7010.1 ?0.4000.6709
0 1 0.9?0.6100.1 ?0.3400.5830
1 0 0.6?0.7010.4 ?0.4000.5806
1 1 0.6?0.6100.4 ?0.3400.5020
27
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
a c ?B(a, c)
0 0 0.6709
0 1 0.5830
1 0 0.5806
1 1 0.5020
f ?G(f )
0.1
? 0.7
a ?C(a )
1 0.67 ?0.58060.33 ?0.50200.554662
0 0.75 ?0.67090.25 ?0.58300.648925
28
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
a c ?B(a, c)
0 0 0.6709
0 1 0.5830
1 0 0.5806
1 1 0.5020
f ?G(f )
0.1
? 0.7
a ?C(a )
1 0.554662
0 0.648925
a P(a, g1)
1 0.3?0.5546620.1663986
0 0.7?0.6489250.4542475
a P(a g1)
1 0.1663986/0.62064610.26811
0 0.4542475/0.62064610.73189
29
Bucket Elimination Algorithm
30
Complexity
• The BuckElim Algorithm can be applied to any
ordering.
• The arity of the function recorded in a bucket
• the numbers of variables appearing in the
processed bucked, excluding the buckets
variable.
• Time and Space complexity is exponentially grow
with a function of arity r.
• The arity is dependent on the ordering.
• How many possible orderings for BNs variables?

31
Determination of the Arity
Consider the ordering AFDCBG.
BucketG
BucketB
1
G
4
BucketC
B
1
,3
C
BucketD
0
,2
D
BucketF
0
,1
F
BucketA
0
A
32
Determination of the Arity
d
Given the ordering, e.g., AFDCBG.
The width of a graph is the maximum width of its
nodes.
w(d) 4
w(d) 4
w(d) width of initial graph for
ordering d. w(d) width of induced graph
for ordering d.
Width of node
Width of node
G
B
C
Induced Graph
D
Initial Graph
F
A
33
Definition of Tree-Width
Goal Finding an ordering with smallest induced
width.
Greedy heuristic and Approximation methods Are
available.
NP-Hard
34
Summary
• The complexity of BuckElim algorithm is dominated
by the time and space needed to process a bucket.
• It is time and space is exponential in number of
bucket variables.
• Induced width bounds the arity of bucket
functions.

35
Exercises
• Use BuckElim to evaluate P(ab1) with the
following two ordering
• d1ACBFDG
• d2AFDCBG

Give the details and make some conclusion.
How to improve the algorithm?
36
Bayesian Networks Bucket Elimination Algorithm
• Most Probable Explanation (MPE)
• ???????
• ?????????

37
MPE
Goal
evidence
38
MPE
Goal
39
Notations
40
MPE
Let
41
MPE
Some terms involve xn, some terms not.
Xn is conditioned by its parents.
Xn conditions its children.
42
MPE
xn appears in these CPTs
Not conditioned by xn
Conditioned by xn
Itself
43
MPE
Process the next bucket recursively.
Eliminate variable xn at Bucketn.
44
Example
45
Example
Consider ordering ACBFDG
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
46
Bucket Elimination Algorithm
47
Exercise
Consider ordering ACBFDG
48
Bayesian Networks Bucket Elimination Algorithm
• Maximum
• A Posteriori (MAP)
• ???????
• ?????????

49
MAP
Given a belief network, a subset of hypothesized
variables A(A1, , Ak), and evidence Ee, the
goal is to determine
50
Example
Hypothesis (Decision) Variables
g 1
51
MAP
Ordering
Some of them may be observed
52
MAP
53
MAP
54
MAP
Bucket Elimination for belief updating
Bucket Elimination for MPE
55
Bucket Elimination Algorithm
56
Example
Consider ordering CBAFDG
BucketG
BucketD
BucketF
BucketA
BucketB
BucketC
57
Exercise
Consider ordering CBAFDG
BucketG
BucketD
BucketF
BucketA
Give the detail
BucketB
BucketC