Dynamic Bayesian Networks

1
Dynamic Bayesian Networks

• Patricia Rosario

2
Table of Contents

• Motivation
• Bayesian Networks
• Bayes Theorem
• Dynamic Bayesian Networks
• Representation
• Basic Inference
• Approximate Inference
• Special Cases
• Applications
• Conclusions
• BLOg
• References

3
Motivation

• Bayesian Networks offer a number of
  well-documented advantages for the representation
  and processing of knowledge and uncertainty.
• BN can be understood by humans as well as learned
  from observed data
• Real time algorithms for reasoning under
  uncertainty exist.

4
Bayesian Networks
Graphical representations of joint distributions
Static world, each random variable has a single
fixed value.
5
Bayes Theorem
Mathematical formula used for calculating
conditional probabilities. Develop by the
mathematician and theologian Thomas Bayes
(published in 1763)
6
Dynamic Bayesian Network

• How can we model dynamic situations?

The process of change can de viewed as a set of
slices, each of which describes the state of the
world at a particular time.
Set of unobservable state variables at time t.
Set of observable evidence variables at time t.
The term dynamic means we are modeling a
dynamic system, not that the network structure
changes over time.
7
DBN - Representation

• Example
• Target Is it raining today?

next step specify dependencies among the
variables.
8
DBN - Representation

• Problem

• Necessity to specify an unbounded number of
  conditional probability table, one for each
  variable in each slice,
• Each one might involve an unbounded number of
  parents.

• Solution

• Assume that change in the world state are caused
  by a stationary process (unmoving process over
  time).

is the same for all t
9
DBN - Representation

• Solution cont.

2. Use Markov assumption - The current state
depends on only in a finite history of previous
states. Using the first-order Markov process
Transition Model
In addition to restricting the parents of the
state variable Xt, we must restrict the parents
of the evidence variable Et
Sensor Model
10
Dynamic Bayesian Networks
Bayesian network structure corresponding to a
first-order of Markov process with state defined
by the variables Xt.
A second order of Markov process
11
Dynamic Bayesian Networks
Bayesian network structure describing the
umbrella world. The transition model is
and
the sensor model is
12
Dynamic Bayesian Networks

• There are two possible fixes if the approximation
  is too inaccurate
• Increasing the order of the Markov process model.
  For example, adding as a parent of
  , which might give slightly more accurate
  predictions.
• Increasing the set of state variables. For
  example, adding to allow to
  incorporate historical records of rainy seasons,
  or adding ,
  and Pressure to allow to use a
  physical model of rainy conditions.

13
DBN Basic Inference

• Filtering or monitoring
• Prediction
• Smoothing or hindsight
• Most likely explanation
• The details of how to perform these computations
  depend on which model and which algorithm is used.

14
DBN Basic Inference

• Filtering or Monitoring
• Compute the belief state - the posterior
  distribution over the current state, given all
  evidence to date.

Filtering is what a rational agent needs to do in
order to keep track of the current state so that
the rational decisions can be made.
15
DBN Basic Inference

• Filtering cont.

Given the results of filtering up to time t, one
can easily compute the result for t1 from the
new evidence
(for some function f)
(dividing up the evidence)
(using Bayes Theorem)
(by the Marcov propertyof evidence)
a is a normalizing constant used to make
probabilities sum up to 1.
16
DBN Basic Inference

• Filtering cont.

The second term represents a
one-step prediction of the next step, and the
first term updates this with
the new evidence. Now we obtain the one-step
prediction for the next step by conditioning on
the current state Xt
(using the Marcov property)
17
DBN Basic Inference

• Illustration for two steps in the Umbrella
  example
• On day 1, the umbrella appears so U1true. The
  prediction from t0 to t1 is

and updating it with the evidence for t1 gives

• On day 2, the umbrella appears so U2true. The
  prediction from t1 to t2 is

and updating it with the evidence for t2 gives
18
DBN Basic Inference

• Prediction
• Compute the posterior distribution over the
  future state, given all evidence to date.

for some kgt0
The task of prediction can be seen simply as
filtering without the addition of new evidence.
19
DBN Basic Inference

• Smoothing or hindsight
• Compute the posterior distribution over the past
  state, given all evidence up to the present.

for some k such that 0 k lt t.
Hindsight provides a better estimate of the state
than was available at the time, because it
incorporates more evidence.
20
DBN Basic Inference

• Most likely explanation
• Compute the sequence of states that is most
  likely to have generated a given sequence of
  observation.

Algorithms for this task are useful in many
applications, including speech recognition.
21
DBN Basic Inference

• Most likely explanation cont.

There exist a recursive relationship between the
most likely paths to each state Xt1 and the most
likely paths to each state Xt. This relationship
can be write as an equation connecting the
probabilities of the paths
22
DBN Approximate inference

• Even though we can use DBN to represent very
  complex temporal process with many sparsely
  connected variables, we cannot reason efficiently
  and exact about those process.
• Thus, we must fall back on approximate methods.

23
DBN Approximate inference

• Particle Filtering Algorithms

Focus the set of samples on the high probability
regions of the state space, throwing away samples
that have very low weight, according to the
observation, while multiplying those that have
high weight. In that way the population of
samples will stay reasonably close to reality.
24
Particle Filtering Algorithm
25
Dynamic Bayesian Networks

• In addition to these tasks, methods are needed
  for learning the transition and sensor models
  from observation.
• Learning can be done by inference, where
  inference provides an estimate of what
  transitions actually occurred and of what states
  generated the sensor readings. These estimates
  can be used to update the models.
• The updated models provide new estimates, and the
  process iterates to convergence.

26
Dynamic Bayesian Networks

• Learning requires the full smoothing inference,
  rather than filtering, because it provides better
  estimates of the state of the process.
• Learning the parameters of a BN is done using
  Expectation Maximization (EM) Algorithms.
  Iterative optimization method to estimate some
  unknowns parameters.

27
DBN Special Cases

• Hidden Markov Model (HMMs)
• Temporal probabilistic model in which the state
  of the process is described by a single discrete
  random variable. (The simplest kind of DBN )
• Kalman Filter Models (KFMs)
• Estimate the state of a physical system from
  noisy observations over time. Also known as
  linear dynamical systems (LDSs).

28
Dynamic Bayesian Networks

• Conclusions
• A DBN is a Bayesian network that represents a
  temporal probability model.
• In general each slice of a DBN can have any
  number of state variables and evidence
  variables .
• For simplicity it can be assume that the
  variables and their links are exactly replicated
  from slice to slice and that the DBN represents a
  first order Markov process.

29
DBN - Applications

• Areas such as Academics, Biology, Business and
  Finance, Computer Games, Computer Vision,
  Computer Software, Medicine, Planning,
  Psychology, Scheduling, Speech Recognition, etc.
  
• The most widely used are the ones embedded in
  Microsoft's products, including the Answer Wizard
  of Office 95, the Office Assistant (the bouncy
  paperclip guy) of Office 97, and over 30
  Technical Support Troubleshooters.
  (research.microsoft.com)
• Another interesting fielded application is the
  Vista system, developed by Eric Horvitz. The
  Vista system is a decision-theoretic system that
  has been used at NASA Mission Control Center in
  Houston for several years. (research.microsoft.com
  /research/dtg/horvitz/vista.htm)

30
BLOG Bayesian LOGic

• A formal, compact and intuitive language for
  defining probability distribution models over
  outcomes with varying sets of objects (worlds
  with unknowns objects and identity uncertainty).

31
DBN - References

• Russel, S. and Norvig, P. Artificial
  Intelligence A Modern Approach. Second Edition.
  Prentice Hall. 2003.
• Murphy, K. Dynamic Bayesian Networks
  Representation, inference and Learning. Phd
  Thesis. UC Berkeley, Computer Science Division,
  July 2002.
• Neopolotan, R. Learning Bayesian Networks.
  Prentice Hall. 2003.
• http//en.wikipedia.org/wiki/Bayesian_network
• http//www.cs.ubc.ca/murphyk/Bayes/bnintro.htmli
  nfer