Variational Bayes Model Selection for Mixture Distribution - PowerPoint PPT Presentation

1 / 18
About This Presentation
Title:

Variational Bayes Model Selection for Mixture Distribution

Description:

Automatic Relevance Determination (ARD) Determination of hyperparameters: Likelihood: ... Automatic Relevance Determination (ARD) State elimination: if , ... – PowerPoint PPT presentation

Number of Views:140
Avg rating:3.0/5.0
Slides: 19
Provided by: JSH55
Category:

less

Transcript and Presenter's Notes

Title: Variational Bayes Model Selection for Mixture Distribution


1
Variational Bayes Model Selectionfor Mixture
Distribution
Authors Adrian Corduneanu Christopher M. Bishop
Presented by Shihao Ji Duke University Machine
Learning Group Jan. 20, 2006
2
Outline
  • Introduction model selection
  • Automatic Relevance Determination (ARD)
  • Experimental Results
  • Application to HMMs

3
Introduction
  • Cross validation
  • Bayesian approaches
  • MCMC and Laplace approximation
  • (Traditional) variational method
  • (Type II) variational method

4
Automatic Relevance Determination (ARD)
  • relevance vector regression
  • Given a dataset , we
    assume is Gaussian

Likelihood
Prior
Posterior
Determination of hyperparameters
Type II ML
5
Automatic Relevance Determination (ARD)
  • mixture of Gaussian
  • Given an observed dataset
    , we assume each data point is drawn
  • independently from a mixture of Gaussian
    density

Likelihood
Prior
Posterior
VB
Determination of mixing coefficients
Type II ML
6
Automatic Relevance Determination (ARD)
  • model selection

Bayesian method
,
Component elimination if
,
i.e.,
7
Experimental Results
  • Bayesian method vs. cross-validation

600 points drawn from a mixture of 5 Gaussians.
8
Experimental Results
  • Component elimination

Initially the model had 15 mixtures, finally was
pruned down to 3 mixtures
9
Experimental Results
10
Automatic Relevance Determination (ARD)
  • hidden Markov model
  • Given an observed dataset
    , we assume each data sequence is
  • generated independently from an HMM

Likelihood
Prior
Posterior
VB
Determination of p and A
Type II ML
11
Automatic Relevance Determination (ARD)
  • model selection

Bayesian method
,
State elimination if ,
Define -- visiting frequency
where
12
Experimental Results (1)
13
(No Transcript)
14
Experimental Results (2)
15
(No Transcript)
16
Experimental Results (3)
17
(No Transcript)
18
Questions?
Write a Comment
User Comments (0)
About PowerShow.com