Dependent%20Hierarchical%20Normalized%20Random%20Measures%20for%20Dynamic%20Topic%20Modeling%20%20Changyou%20Chen,%20Nan%20Ding%20and%20Wray%20Buntine%20ICML%202012

1
Dependent Hierarchical Normalized Random Measures
for Dynamic Topic Modeling Changyou Chen, Nan
Ding and Wray BuntineICML 2012

• Presented by Mingyuan Zhou
• Duke University, ECE
• October 24, 2012

2
Introduction

• NRM normalized random measures with independent
  increments
• Superposition, subsampling and point transition
  of NRM
• Dependent hierarchical NRM
• Dynamic topic modeling

3
Normalized Random Measures

• Poisson process
• Completely random measures (CRM)

4
Normalized Random Measures

• Completely random measures (CRM)

5
Normalized Random Measures

• Slice sampling NRMs
• Ref Griffin, J.E. and Walker, S.G. Posterior
  simulation of normalized random measure mixtures.
  J. Comput. Graph. Stat., 2011.

6
Normalized Random Measures

• Normalized generalized gamma process

7
Dynamic topic modeling with dependent
hierarchical NRMs

• Ideas
• Inherit topics from the previous time frame
  through three dependency operators
• Superposition
• Subsampling
• Point transition
• Generate new topics

8
Dynamic topic modeling with dependent
hierarchical NRMs

9
Dynamic topic modeling with dependent
hierarchical NRMs

10
Dynamic topic modeling with dependent
hierarchical NRMs

11
Dynamic topic modeling with dependent
hierarchical NRMs

• Properties of the dependence operators

12
Dynamic topic modeling with dependent
hierarchical NRMs

• Reformulated model

13
Dynamic topic modeling with dependent
hierarchical NRMs

• Original and reformulated model

14
Sampling

• Sampling under the Chinese restaurant metaphor

15
Sampling

• Sampling under the Chinese restaurant metaphor

16
Sampling
17
Sampling
18
Sampling
19
Experiments

• Power-law in the NGG

20
Experiments

21
Experiments

22
Experiments

23
Experiments

24
Experiments

25
Conclusions