Bayesian Networks Bucket Elimination Algorithm - PowerPoint PPT Presentation

Loading...

PPT – Bayesian Networks Bucket Elimination Algorithm PowerPoint presentation | free to download - id: 4cc76f-OGQ5M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Bayesian Networks Bucket Elimination Algorithm

Description:

Bucket Elimination The algorithm will be used as a framework for various probabilistic inferences on Bayesian ... Bayesian Networks Variable Elimination Algorithm ... – PowerPoint PPT presentation

Number of Views:81
Avg rating:3.0/5.0
Slides: 58
Provided by: TaiWe9
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Bayesian Networks Bucket Elimination Algorithm


1
Bayesian NetworksBucket Elimination Algorithm
  • ??????
  • ???????
  • ?????????

2
Content
  • Basic Concept
  • Belief Updating
  • Most Probable Explanation (MPE)
  • Maximum A Posteriori (MAP)

3
Bayesian NetworksBucket Elimination Algorithm
  • Basic Concept
  • ???????
  • ?????????

4
Satisfiability
Given a statement of clauses (in disjunction
normal form), the satisfiability problem is to
determine whether there exists a truth assignment
to make the statement true.
Examples
Satisfiable
ATrue, BTrue, CFalse, DFalse
Satisfiable?
5
Resolution
can be true if and only if
can be true.
?
?
unsatisfiable
6
Direct Resolution
Example
Given a set of clauses
and an order dABCD
Set initial buckets as follows
7
Direct Resolution
Because no empty clause (???) is resulted, the
statement is satisfiable.
How to get a truth assignment?
8
Direct Resolution
9
Direct Resolution
10
Queries on Bayesian Networks
  • Belief updating
  • Finding the most probable explanation (mpe)
  • Given evidence, finding a maximum probability
    assignment to the rest of variables.
  • Maximizing a posteriori hypothesis (map)
  • Given evidence, finding an assignment to a subset
    of hypothesis variables that maximize their
    probability.
  • Maximizing the expected utility of the problem
    (meu)
  • Given evidence and utility function, finding a
    subset of decision variables that maximize the
    expected utility.

11
Bucket Elimination
  • The algorithm will be used as a framework for
    various probabilistic inferences on Bayesian
    Networks.

12
Preliminary Elimination Functions
Given a function h defined over subset of
variables S, where X ? S,
Eliminate parameter X from h
Defined over U S X.
13
Preliminary Elimination Functions
Given a function h defined over subset of
variables S, where X ? S,
14
Preliminary Elimination Functions
Given function h1,, hn defined over subset of
variables S1,, Sn, respectively,
Defined over
15
Preliminary Elimination Functions
Given function h1,, hn defined over subset of
variables S1,, Sn, respectively,
16
Bayesian NetworksBucket Elimination Algorithm
  • Belief Updating
  • ???????
  • ?????????

17
Goal
Normalization Factor
18
Basic Concept of Variable Elimination
Example
19
Basic Concept of Variable Elimination
Example
20
Basic Concept of Variable Elimination
?G(f)
?D(a, b)
?F(b, c)
?B(a, c)
?C(a)
21
Basic Concept of Variable Elimination
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
22
Basic Concept of Variable Elimination
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
23
Basic Concept of Variable Elimination
f ?G(f )
0.1
? 0.7
24
Basic Concept of Variable Elimination
f ?G(f )
0.1
? 0.7
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
25
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
f ?G(f )
0.1
? 0.7
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
26
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
f ?G(f )
0.1
? 0.7
a c ?B(a, c)
0 0 0.9?0.7010.1 ?0.4000.6709
0 1 0.9?0.6100.1 ?0.3400.5830
1 0 0.6?0.7010.4 ?0.4000.5806
1 1 0.6?0.6100.4 ?0.3400.5020
27
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
a c ?B(a, c)
0 0 0.6709
0 1 0.5830
1 0 0.5806
1 1 0.5020
f ?G(f )
0.1
? 0.7
a ?C(a )
1 0.67 ?0.58060.33 ?0.50200.554662
0 0.75 ?0.67090.25 ?0.58300.648925
28
Basic Concept of Variable Elimination
a b ?D(a, b)
0 0 1
0 1 1
1 0 1
1 1 1
b c ?F(b, c)
0 0 0.701
0 1 0.610
1 0 0.400
1 1 0.340
a c ?B(a, c)
0 0 0.6709
0 1 0.5830
1 0 0.5806
1 1 0.5020
f ?G(f )
0.1
? 0.7
a ?C(a )
1 0.554662
0 0.648925
a P(a, g1)
1 0.3?0.5546620.1663986
0 0.7?0.6489250.4542475
a P(a g1)
1 0.1663986/0.62064610.26811
0 0.4542475/0.62064610.73189
29
Bucket Elimination Algorithm
30
Complexity
  • The BuckElim Algorithm can be applied to any
    ordering.
  • The arity of the function recorded in a bucket
  • the numbers of variables appearing in the
    processed bucked, excluding the buckets
    variable.
  • Time and Space complexity is exponentially grow
    with a function of arity r.
  • The arity is dependent on the ordering.
  • How many possible orderings for BNs variables?

31
Determination of the Arity
Consider the ordering AFDCBG.
BucketG
BucketB
1
G
4
BucketC
B
1
,3
C
BucketD
0
,2
D
BucketF
0
,1
F
BucketA
0
A
32
Determination of the Arity
d
Given the ordering, e.g., AFDCBG.
The width of a graph is the maximum width of its
nodes.
w(d) 4
w(d) 4
w(d) width of initial graph for
ordering d. w(d) width of induced graph
for ordering d.
Width of node
Width of node
G
B
C
Induced Graph
D
Initial Graph
F
A
33
Definition of Tree-Width
Goal Finding an ordering with smallest induced
width.
Greedy heuristic and Approximation methods Are
available.
NP-Hard
34
Summary
  • The complexity of BuckElim algorithm is dominated
    by the time and space needed to process a bucket.
  • It is time and space is exponential in number of
    bucket variables.
  • Induced width bounds the arity of bucket
    functions.

35
Exercises
  • Use BuckElim to evaluate P(ab1) with the
    following two ordering
  • d1ACBFDG
  • d2AFDCBG

Give the details and make some conclusion.
How to improve the algorithm?
36
Bayesian NetworksBucket Elimination Algorithm
  • Most Probable Explanation (MPE)
  • ???????
  • ?????????

37
MPE
Goal
evidence
38
MPE
Goal
39
Notations
40
MPE
Let
41
MPE
Some terms involve xn, some terms not.
Xn is conditioned by its parents.
Xn conditions its children.
42
MPE
xn appears in these CPTs
Not conditioned by xn
Conditioned by xn
Itself
43
MPE
Process the next bucket recursively.
Eliminate variable xn at Bucketn.
44
Example
45
Example
Consider ordering ACBFDG
BucketG
BucketD
BucketF
BucketB
BucketC
BucketA
46
Bucket Elimination Algorithm
47
Exercise
Consider ordering ACBFDG
48
Bayesian NetworksBucket Elimination Algorithm
  • Maximum
  • A Posteriori (MAP)
  • ???????
  • ?????????

49
MAP
Given a belief network, a subset of hypothesized
variables A(A1, , Ak), and evidence Ee, the
goal is to determine
50
Example
Hypothesis (Decision) Variables
g 1
51
MAP
Ordering
Some of them may be observed
52
MAP
53
MAP
54
MAP
Bucket Elimination for belief updating
Bucket Elimination for MPE
55
Bucket Elimination Algorithm
56
Example
Consider ordering CBAFDG
BucketG
BucketD
BucketF
BucketA
BucketB
BucketC
57
Exercise
Consider ordering CBAFDG
BucketG
BucketD
BucketF
BucketA
Give the detail
BucketB
BucketC
About PowerShow.com