I Spent an Interesting Evening Recently with a Grain of Salt

1
"I Spent an Interesting Evening Recently with a
Grain of Salt "
"As I've commented before, really relating to
someone involves standing next to impossible."
"One morning I shot an elephant in my arms and
kissed him. So it was too small for a pill? "
"Oh, sorry. Nevermind. I am afraid of it
becoming another island in a nice suit."
The power of the 2nd order Markov Chain. After
Claude Shannon. With thanks to Alyosha Efros.
2
Markov chains and processes
1st order Markov chain
2nd order Markov chain
1st order with stochastic observations -- HMM
3
HMM analysis of speech
4
.. Markov processes

• Markov Random Field eg for texture modelling
  (section 4)
• intractable cf. Markov chain

Intractable because of

• bidirectionality
• twodimensionality

5
Markov Chain joint distribution
1st order Markov chain
Joint distribution
Markov property, 1st order
-- process distribution, so joint
Matrix
-- reversible
6
Markov Chain transition diagram
-- for
1st order Markov chain
Process model
7
Hidden Markov Model (HMM)
emission/observation
Observer distribution
eg
Examples
speech
phoneme
spectral o/p
disparity
pixel pair
stereo
viseme
motion field
gesture
Hidden state
Observations
8
HMMs three famous problems (Rabiner, 89,93)

• Model classification
• MAP state sequence estimation
• MLE model parameter learning

for various
emission
transition
9
HMMs three famous solutions

• Model classification
• MAP state sequence estimation
• MLE model parameter learning

Forward algorithm
Viterbi algorithm (DP)
Baum-Welch algorithm (DP)
10
1. Model classification Forward algorithm
Forward probability
Iteration
Model likelihood
11
2. MAP state sequence Viterbi algorithm (DP)
Partial posterior probability
Iteration
MAP sequence
for efficiency (additivity), define instead
12
3. MLE for HMM parameter learning Baum-Welch
would like
so do EM maximising
where
emission
transition
substituting expected values from
and
using FB (see Belief Propagation), and iterate
13
Markov Chain Bidirectional process
1st order Markov Random Field (MRF)
Joint distribution
equivalent to bidirectional, factorised, joint
distribution
? stereo scanlines
14
Low-level view interpolation
StereoReconstruction
Left camera
Right camera
Cyclopean ground truth
15
Epipolar matchingas optimal path finding
(Ohta Kanade, 1985 Cox, Hingorani Rao, 1996)
Min cost path
16
Elaborated hidden variables ? constraints
(Criminisi, Shotton, Blake, Rother, Torr 2003)
3-move, 1-plane DP
Improvement 3-plane DP explicit occlusion
Much better quality streaks and halo removed a
little temporal flicker
Artefacts streaks, halo, temporal flicker
17
Elaborated hidden variables ? constraints
3-plane DP explicit occlusion labelling
18
Probabilistic stereo
(Belhumeur 1996)

• Entire disparity distribution ambiguous
  matches
• MAP invariant to
• State lifetimes (Markov chain) from
• Learned observation distribution

19
Probabilistic model
Path Cost
Posterior distribution
Bayes
Markov chain
Observations
20
Probabilistic model -- well formed?

• Observations
• Space of paths
• unconstrained
• cyclopean ordering on k
• DG limit

each observable pixel/patch
counted singly
In
not too large
DP complexity
Image width N
21
Cyclopean Coordinates
(Belhumeur 1996)

• Cyclopean monotonicity
• HV moves

22
FB disparity distributions
disparity
23
Video Rewrite (C. Bregler, Siggraph 97)

• Phoneme-labelled speech data from TIMIT
• Learn 3-state HMM for each of 61 phonemes
• Use known groupings into 26 visemes
• Model match viseme triphones in training
• video (coarticulation)
• Synthesis extract video cliplets corresponding
  to
• desired sequence of viseme triphones

/SIL-T-IY/ /T-IY-P/ /IY-P-AA/ /P-AA-T/
/AA-T-SIL/
Teapot
24
Video Rewrite