Loading...

PPT – Bayesian Networks Bucket Elimination Algorithm PowerPoint presentation | free to download - id: 4cc76f-OGQ5M

The Adobe Flash plugin is needed to view this content

Bayesian NetworksBucket Elimination Algorithm

- ??????
- ???????
- ?????????

Content

- Basic Concept
- Belief Updating
- Most Probable Explanation (MPE)
- Maximum A Posteriori (MAP)

Bayesian NetworksBucket Elimination Algorithm

- Basic Concept
- ???????
- ?????????

Satisfiability

Given a statement of clauses (in disjunction

normal form), the satisfiability problem is to

determine whether there exists a truth assignment

to make the statement true.

Examples

Satisfiable

ATrue, BTrue, CFalse, DFalse

Satisfiable?

Resolution

can be true if and only if

can be true.

?

?

unsatisfiable

Direct Resolution

Example

Given a set of clauses

and an order dABCD

Set initial buckets as follows

Direct Resolution

Because no empty clause (???) is resulted, the

statement is satisfiable.

How to get a truth assignment?

Direct Resolution

Direct Resolution

Queries on Bayesian Networks

- Belief updating
- Finding the most probable explanation (mpe)
- Given evidence, finding a maximum probability

assignment to the rest of variables. - Maximizing a posteriori hypothesis (map)
- Given evidence, finding an assignment to a subset

of hypothesis variables that maximize their

probability. - Maximizing the expected utility of the problem

(meu) - Given evidence and utility function, finding a

subset of decision variables that maximize the

expected utility.

Bucket Elimination

- The algorithm will be used as a framework for

various probabilistic inferences on Bayesian

Networks.

Preliminary Elimination Functions

Given a function h defined over subset of

variables S, where X ? S,

Eliminate parameter X from h

Defined over U S X.

Preliminary Elimination Functions

Given a function h defined over subset of

variables S, where X ? S,

Preliminary Elimination Functions

Given function h1,, hn defined over subset of

variables S1,, Sn, respectively,

Defined over

Preliminary Elimination Functions

Given function h1,, hn defined over subset of

variables S1,, Sn, respectively,

Bayesian NetworksBucket Elimination Algorithm

- Belief Updating
- ???????
- ?????????

Goal

Normalization Factor

Basic Concept of Variable Elimination

Example

Basic Concept of Variable Elimination

Example

Basic Concept of Variable Elimination

?G(f)

?D(a, b)

?F(b, c)

?B(a, c)

?C(a)

Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

Basic Concept of Variable Elimination

f ?G(f )

0.1

? 0.7

Basic Concept of Variable Elimination

f ?G(f )

0.1

? 0.7

a b ?D(a, b)

0 0 1

0 1 1

1 0 1

1 1 1

Basic Concept of Variable Elimination

a b ?D(a, b)

0 0 1

0 1 1

1 0 1

1 1 1

f ?G(f )

0.1

? 0.7

b c ?F(b, c)

0 0 0.701

0 1 0.610

1 0 0.400

1 1 0.340

Basic Concept of Variable Elimination

a b ?D(a, b)

0 0 1

0 1 1

1 0 1

1 1 1

b c ?F(b, c)

0 0 0.701

0 1 0.610

1 0 0.400

1 1 0.340

f ?G(f )

0.1

? 0.7

a c ?B(a, c)

0 0 0.9?0.7010.1 ?0.4000.6709

0 1 0.9?0.6100.1 ?0.3400.5830

1 0 0.6?0.7010.4 ?0.4000.5806

1 1 0.6?0.6100.4 ?0.3400.5020

Basic Concept of Variable Elimination

a b ?D(a, b)

0 0 1

0 1 1

1 0 1

1 1 1

b c ?F(b, c)

0 0 0.701

0 1 0.610

1 0 0.400

1 1 0.340

a c ?B(a, c)

0 0 0.6709

0 1 0.5830

1 0 0.5806

1 1 0.5020

f ?G(f )

0.1

? 0.7

a ?C(a )

1 0.67 ?0.58060.33 ?0.50200.554662

0 0.75 ?0.67090.25 ?0.58300.648925

Basic Concept of Variable Elimination

a b ?D(a, b)

0 0 1

0 1 1

1 0 1

1 1 1

b c ?F(b, c)

0 0 0.701

0 1 0.610

1 0 0.400

1 1 0.340

a c ?B(a, c)

0 0 0.6709

0 1 0.5830

1 0 0.5806

1 1 0.5020

f ?G(f )

0.1

? 0.7

a ?C(a )

1 0.554662

0 0.648925

a P(a, g1)

1 0.3?0.5546620.1663986

0 0.7?0.6489250.4542475

a P(a g1)

1 0.1663986/0.62064610.26811

0 0.4542475/0.62064610.73189

Bucket Elimination Algorithm

Complexity

- The BuckElim Algorithm can be applied to any

ordering. - The arity of the function recorded in a bucket
- the numbers of variables appearing in the

processed bucked, excluding the buckets

variable. - Time and Space complexity is exponentially grow

with a function of arity r. - The arity is dependent on the ordering.
- How many possible orderings for BNs variables?

Determination of the Arity

Consider the ordering AFDCBG.

BucketG

BucketB

1

G

4

BucketC

B

1

,3

C

BucketD

0

,2

D

BucketF

0

,1

F

BucketA

0

A

Determination of the Arity

d

Given the ordering, e.g., AFDCBG.

The width of a graph is the maximum width of its

nodes.

w(d) 4

w(d) 4

w(d) width of initial graph for

ordering d. w(d) width of induced graph

for ordering d.

Width of node

Width of node

G

B

C

Induced Graph

D

Initial Graph

F

A

Definition of Tree-Width

Goal Finding an ordering with smallest induced

width.

Greedy heuristic and Approximation methods Are

available.

NP-Hard

Summary

- The complexity of BuckElim algorithm is dominated

by the time and space needed to process a bucket. - It is time and space is exponential in number of

bucket variables. - Induced width bounds the arity of bucket

functions.

Exercises

- Use BuckElim to evaluate P(ab1) with the

following two ordering - d1ACBFDG
- d2AFDCBG

Give the details and make some conclusion.

How to improve the algorithm?

Bayesian NetworksBucket Elimination Algorithm

- Most Probable Explanation (MPE)
- ???????
- ?????????

MPE

Goal

evidence

MPE

Goal

Notations

MPE

Let

MPE

Some terms involve xn, some terms not.

Xn is conditioned by its parents.

Xn conditions its children.

MPE

xn appears in these CPTs

Not conditioned by xn

Conditioned by xn

Itself

MPE

Process the next bucket recursively.

Eliminate variable xn at Bucketn.

Example

Example

Consider ordering ACBFDG

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

Bucket Elimination Algorithm

Exercise

Consider ordering ACBFDG

Bayesian NetworksBucket Elimination Algorithm

- Maximum
- A Posteriori (MAP)
- ???????
- ?????????

MAP

Given a belief network, a subset of hypothesized

variables A(A1, , Ak), and evidence Ee, the

goal is to determine

Example

Hypothesis (Decision) Variables

g 1

MAP

Ordering

Some of them may be observed

MAP

MAP

MAP

Bucket Elimination for belief updating

Bucket Elimination for MPE

Bucket Elimination Algorithm

Example

Consider ordering CBAFDG

BucketG

BucketD

BucketF

BucketA

BucketB

BucketC

Exercise

Consider ordering CBAFDG

BucketG

BucketD

BucketF

BucketA

Give the detail

BucketB

BucketC