Bayesian Networks Lecture 1: Basics and Knowledge-Based Construction

1
Bayesian Networks Lecture 1Basics and
Knowledge-Based Construction
Lecture 1 is based on David Heckermans Tutorial
slides. (Microsoft Research)
Requirements 50 home works 50 Exam or a
project
2
What I hope you will get out ofthis course...

• What are Bayesian networks?
• Why do we use them?
• How do we build them by hand?
• How do we build them from data?
• What are some applications?
• What is their relationship to other models?
• What are the properties of conditional
  independence that make these models appropriate?
• Usage in genetic linkage analysis

3
Applications of hand-built Bayes Nets

• Answer Wizard 95, Office Assistant 97,2000
• Troubleshooters in Windows 98
• Lymph node pathology
• Trauma care
• NASA mission control

Some Applications of learned Bayes Nets

• Clustering users on the web (MSNBC)
• Classifying Text (spam filtering)

4
Some factors that support intelligence

• Knowledge representation
• Reasoning
• Learning / adapting

5
Artificial Intelligence
6
Artificial Intelligence is better than none !
7
Artificial Intelligence is better than ours !
8
Outline for today

• Basics
• Knowledge-based construction
• Probabilistic inference
• Applications of hand-built BNs at Microsoft

9
Bayesian Networks History

• 1920s Wright -- analysis of crop failure
• 1950s I.J. Good -- causality
• Early 1980s Howard and Matheson, Pearl
• Other names
• directed acyclic graphical (DAG) models
• belief networks
• causal networks
• probabilistic networks
• influence diagrams
• knowledge maps

10
Bayesian Network
p(f)
p(b)
p(gf,b)
p(tb)
p(sf,t)
Directed Acyclic Graph, annotated with prob
distributions
11
BN structure Definition

• Missing arcs encode independencies such that

12
Independencies in a Bayes net
Example
Many other independencies are entailed by ()
can be read from the graph using d-separation
(Pearl)
13
Explaining Away and Induced Dependencies
"explaining away" "induced dependencies"
14
Local distributions
Table p(SyTn,Fe) 0.0 p(SyTn,Fn)
0.0 p(SyTy,Fe) 0.0 p(SyTy,Fn) 0.99
15
Local distributions
Tree
16
Lots of possibilities for a local distribution...

• y discrete node any probabilistic classifier
• Decision tree
• Neural net
• y continuous node any probabilistic regression
  model
• Linear regression with Gaussian noise
• Neural net

17
Naïve Bayes Classifier
discrete
18
Hidden Markov Model
discrete, hidden
H1
H2
H3
H4
H5
...
...
X1
X2
X3
X4
X5
observations
19
Feed-Forward Neural Network
X1
X1
X1
inputs
hidden layer
sigmoid
Y1
Y2
Y3
outputs (binary)
sigmoid
20
Outline

• Basics
• Knowledge-based construction
• Probabilistic inference
• Decision making
• Applications of hand-built BNs at Microsoft

21
Building a Bayes net by hand(ok, now we're
starting to be Bayesian)

• Define variables
• Assess the structure
• Assess the local probability distributions

22
What is a variable?

• Collectively exhaustive, mutually exclusive values

Error Occured
No Error
23
Clarity Test Is the variable knowable in
principle

• Is it raining? Where, when, how many inches?
• Is it hot? T ? 100F , T lt 100F
• Is users personality dominant or submissive?
  numerical result of standardized personality
  test

24
Assessing structure(one approach)

• Choose an ordering for the variables
• For each variable, identify parents Pai such that

25
Example
Fuel
26
Example
Fuel
p(f)
27
Example
Fuel
p(bf)p(b)
p(f)
28
Example
Fuel
p(bf)p(b)
p(f)
p(tb,f)p(tb)
29
Example
Fuel
p(bf)p(b)
p(f)
p(tb,f)p(tb)
p(gf,b,t)p(gf,b)
30
Example
p(bf)p(b)
p(f)
p(tb,f)p(tb)
p(gf,b,t)p(gf,b)
p(sf,b,t,g)p(sf,t)
p(f,b,t,g,s) p(f) p(b) p(tb) p(gf,b) p(sf,t)
31
Why is this the wrong way?Variable order can be
critical
32
A better wayUse causal knowledge
33
Conditional Independence Simplifies Probabilistic
Inference
34
Online Troubleshooters
35
Define Problem
36
Gather Information
37
Get Recommendations
38
Portion of BN for print troubleshooting
(see Breese Heckerman, 1996)
39
Office Assistant 97
40
Lumière Project
(see Horvitz, Breese, Heckerman, Hovel Rommelse
1998)

Users Goals
Users Needs
User Activity
41
Studies with Human Subjects

• Wizard of OZ experiments at MS Usability Labs

User Actions
Typed Advice
Expert Advisor
Inexperienced user
42
Activities with Relevance to Users Needs

Several classes of evidence

• Search e.g., menu surfing
• Introspection e.g., sudden pause, slowing of
  command stream
• Focus of attention e.g, selected objects
• Undesired effects e.g., command/undo, dialogue
  opened and cancelled
• Inefficient command sequences
• Goal-specific sequences of actions

43
Summary so far

• Bayes nets are useful because...
• They encode independence explicitly
• more parsimonious models
• efficient inference
• They encode independence graphically
• Easier explanation
• Easier encoding
• They sometimes correspond to causal models
• Easier explanation
• Easier encoding
• Modularity leads to easier maintenance

44
Teenage Bayes
MICRONEWS 97 Microsoft Researchers Exchange
Brainpower with Eighth-grader Teenager Designs
Award-Winning Science Project .. For her science
project, which she called "Dr. Sigmund
Microchip," Tovar wanted to create a computer
program to diagnose the probability of certain
personality types. With only answers from a few
questions, the program was able to accurately
diagnose the correct personality type 90 percent
of the time.
45
Artificial Intelligence is a promising
fieldalways was, always will be.