Maximum a Posterior

1
Maximum a Posterior

• Presented by ???

2
Maximum a Posterior

• Introduction
• MAP for Discrete HMM
• Prior ?Dirichlet
• MAP for Semi-Continuous HMM
• Prior ?Dirichlet normal-Wishart
• Segmental MAP Estimates
• Conclusion
• Appendix-Matrix Calculus

3
Introduction

• HMM parameter estimators have been derived purely
  from the training observation sequences without
  any prior information included.
• There may be many cases in which the prior
  information about the parameters is available, ex
  previous experience

4
Introduction (cont)
5
Discrete HMM
Definition
6
Discrete HMM
Q-function
7
Discrete HMM
Q-function
8
Discrete HMM
Q-function
??
9
Discrete HMM
R-function
10
Discrete HMM
Q
11
Discrete HMM
Initial probability
12
Discrete HMM
Transition probability
13
Discrete HMM
observation probability
14
Discrete HMM
15
Discrete HMM

• How to choose the initial estimate for
  ?
• One reasonable choice of the initial estimate is
  the mode of the prior density.

16
Discrete HMM

• Whats the mode ?
• So applying Lagrange Multiplier we can easily
  derive above modes.
• Example

17
Discrete HMM

• Another reasonable choice of the initial estimate
  is the mean of the prior density.
• Both are some kind of summarization of the
  available information about the parameters before
  any data are observed.

18
SCHMM
19
SCHMM
independent
20
SCHMM
21
SCHMM
Q-function
22
SCHMM
Q-function
23
SCHMM
24
SCHMM
Initial probability

• Differentiating w.r.t and equate
  it to zero.

25
SCHMM
Transition probability

• Differentiating w.r.t and equate
  it to zero.

26
SCHMM
Mixture weight

• Differentiating w.r.t and equate
  it to zero.

27
SCHMM

• Differentiating w.r.t and equate
  it to zero.
• Differentiating w.r.t and equate
  it to zero.

28
SCHMM

• Full Covariance matrix case

29
SCHMM

• Full Covariance matrix case

30
SCHMM

• Full Covariance matrix case

31
SCHMM

• Full Covariance matrix case

(1)
(2)
(3)
32
SCHMM

• Full Covariance matrix case

33
SCHMM
Full Covariance

• The initial estimate can be chosen as the mode of
  the prior PDF
• And also can be chosen as the mean of the prior
  PDF

34
SCHMM

• Diagonal Covariance matrix case
• Then
• and

35
SCHMM

• Diagonal Covariance matrix case

36
SCHMM
Diagonal Covariance

• Diagonal Covariance matrix case

37
SCHMM
Diagonal Covariance

• Diagonal Covariance matrix case

38
SCHMM

• Diagonal Covariance matrix case

39
SCHMM

• Diagonal Covariance matrix case

(1)
(2)
(3)
40
SCHMM
Diagonal Covariance

• Diagonal Covariance matrix case

41
SCHMM
Diagonal Covariance

• The initial estimate can be chosen as the mode of
  the prior PDF
• And also can be chosen as the mean of the prior
  PDF

42
Segmental MAP Estimates
43
Segmental MAP Estimates
DHMM
44
Segmental MAP Estimates
SCHMM
45
Conclusion

• The important issue of prior density is
  discussed.
• Some application
• Model adaptation, HMM training, IR(?)

46
Appendix-Matrix Calculus(1)

• Notation

47
Appendix-Matrix Calculus(2)

• Properties 1
• proof
• Properties 1 Extension
• proof

48
Appendix-Matrix Calculus(3)

• Properties 2
• proof

49
Appendix-Matrix Calculus(4)

• Properties 3
• proof

50
Appendix-Matrix Calculus(5)

• Properties 4
• proof

51
Appendix-Matrix Calculus(6)

• Properties 5
• proof

52
Appendix-Matrix Calculus(7)

• Properties 6
• proof

53
Appendix-Matrix Calculus(8)

• Some other properties