Representation and Modeling of Natural Scenes

1
Representation and Modeling of Natural
Scenes Ying Nian Wu UCLA Department of Statistics
http//www.stat.ucla.edu/ywu/research/
2
Song Chun Zhu
Stefano Soatto
3
(No Transcript)
4
Wu, Zhu, Liu, IJCV 2000 Zhu, Liu, Wu, PAMI 2000
observed image
synthesized image
5
Malik and Perona, late 80s
6
Image I
Filter response
Filtered image
Histogram
Histogram matching (Heeger and Bergen, mid 90s)
7
Global statistical property
Zhu, Liu, Wu, PAMI 2000
Julesz ensemble
Image lattice
Image universe
8
Local statistical property
Wu, Zhu, Liu, IJCV 2000
Large lattice
Small patch
Julesz ensemble
Markov random field

• Gibbs (1902) equivalence of ensembles
• Exponential family model

9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Olshausen Field Sparse coding
Data a collection of natural image patches
Learning basis
Linear representation
Sparseness of coefficients ? linear bases
Mallat and Zhang matching pursuit Candes and
Donoho curvelets
27
(No Transcript)
28
Two-Level Generative Model
Mixture prior for sparseness
Bell Sejnowski (96) Lewiki Olshausen
(99) Olshausen Millman (00) Pece (01) George
McCulloch (95)
29
Wu, Zhu, Guo, ECCV 2002.
Sketch Model

• Model fitting (EM-type iteration)
• Estimate S based on I and Sketch Model (MCMC)
• Fit Sketch Model on S

• Simplification
• Estimate S from I using matching pursuit (Mallat
  Zhang)
• Fit Sketch Model on S (ignoring c and e)

30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Math representations of sketch
List
Bit-map
Causal model for sketch
Pairwise interactions
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
Soatto,Doretto,Wu, ICCV 2001
48
Soatto,Doretto,Wu, ICCV 2001

• Modeling dynamic scenes
• Data
• Model time series
• Representation principal components (Fourier
  bases)
• Autoregressive model

Fouriers solution to heat equation
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
Knowledge K
World W (W_high, W_low)
P(W K)
P(W I K)
P(I W K)
Image I
Why generative modeling?
Representing knowledge Unsupervised learning of
causes Model selection as explaining away Model
checking by synthesis
Physics model and image-based rendering