Wedgelets: A Multiscale Geometric Representation for Images

1
Wedgelets A Multiscale Geometric Representation
for Images
Michael Wakin Rice University dsp.rice.edu Joint
work with Richard Baraniuk, Hyeokho Choi, Justin
Romberg
2
Computational Harmonic Analysis

• Representation
• Analysis study through structure of
  should extract features of
  interest
• Approximation exploit sparsity of

basis, frame
coefficients
3
Nonlinear Approximation
many small
4
Nonlinear Approximation

• -term approximation use largest
  independently
• Greedy / thresholding

5
Error Approximation Rates
as

• Optimize asymptotic error decay rate

6
1-D Piecewise Smooth Signals

• smooth except for singularities at a finite
  number of 0-D points
• Fourier sinusoids suboptimal greedy
  approximation and extraction
• wavelets optimal greedy approximation
• extract singularity structure

7
2-D Piecewise Smooth Signals

• smooth except for singularities along a finite
  number of smooth 1-D curves
• Challenge analyze/approximate geometric
  structure

geometry
texture
texture
8
Wavelet-based Image Processing

• Standard 2-D tensor product wavelet transform

9
Wavelet Challenges

• Current wavelet methods suboptimal for 2-d
  piecewise smooth functions

• Geometrical info not explicit
• Inefficient -large number of significant WCs
  cluster around edge contours, no matter how
  smooth

10
Wavelets and Cartoons

• Even for a smooth C2 contour, which straightens
  at fine scales

11
2-D Wavelets Poor Approximation

• Even for a smooth C2 contour, which straightens
  at fine scales
• Too many wavelets required!

-term wavelet approximation
not
12
Response A Pursuit of Sparsity

• I. Anisotropic, Directional Frames
• curvelets
• contourlets
• bandelets
• etc

13
Response A Pursuit of Sparsity

• I. Anisotropic, Directional Frames
• curvelets
• contourlets
• bandelets
• etc
• II. Geometric Tilings / Dyadic Partitions
• wedgelets
• beamlets
• platelets
• etc

14
Geometry Model for Cartoons

• Toy model flat regions separated by smooth
  contours
• Goal representation that is
• sparse
• simply modeled
• efficiently computed
• extensible to texture regions (RichB -gt Saturday)

15
2-D Dyadic Partition

• Multiscale analysis
• Zoom in by factor of 2 each scale
• Represent image on each block
• Partition/tiling
• not a basis/frame

16
2-D Dyadic Partition Quadtree

• Each parent node has 4 children at next finer
  scale
• Pruning reflects partitioning
• Leaf nodes decorated with atoms

17
Wedgelet Representation Donoho

• Build a cartoon using wedgelets on dyadic squares
  

18
Wedgelet Representation

• Build a cartoon using wedgelets on dyadic squares
  
• Quad-tree structuredeeper in tree finer
  curve approximation
• Decorate leaves with (r,q)
• parameters

19
Wedgelet Representation

• Prune wedgelet quadtree to approximate local
  geometry (adaptive)
• New problem
• How to find the proper approximation?

20
Wedgelet Inference

• Find representation / prune tree to balance a
  fidelity vs. complexity trade-off

21
Wedgelet Inference

• Find representation / prune tree to balance a
  fidelity vs. complexity trade-off
• For Comp(W) measure the complexity of the
  wedgelet representation(quadtree (r,q))
• Donoho Comp(W) leaves
• NLA-like solution

22
Wedgelet Inference

• Find representation / prune tree to balance a
  fidelity vs. complexity trade-off
• Exponential computational cost to solve brute
  force
• Dynamic programming to the rescue!

23

• distortion

complexity
24

• distortion

Each leaf isindependent!
cost
25

• Decision Keep parent or split into 4 children?

26
Top-Down Greedy

• Label each node of the tree with
• Move down from tree root and prune when

15
15
6
4
6
4
3
27
Bottom-Up Dynamic Programming

• Label each node of the tree with
• Move up from the leaves and prune whenelse

15
15
6
4
6
4
3
3
28
Bottom-Up Dynamic Programming

• Label each node of the tree with
• Move up from the leaves and prune whenelse

15
15
6
4
5
4
3
29
Bottom-Up Dynamic Programming

• Label each node of the tree with
• Move up from the leaves and prune whenelse

15
9
6
4
5
4
3
30
Summary Dynamic Programming

• Simple recursive bottom-up algorithm
• Efficient O(N) for N-node tree
• Finds optimal solution provided cost is
  additivewith independent components

31
Wedgelet Inference

• Find representation / prune tree to balance a
  fidelity vs. complexity trade-off
• Optimal approximation
• Near optimal rate distortion decay
• Near optimal estimation Donoho

32
Leaves Complexity Penalty

• Accounts for wedgelet partition size,
  but not wedgelet orientation

leaves
leaves
smooth simple
rough complex
33
Multiscale Geometry Model (MGM)
34
Multiscale Geometry Model (MGM)

• Decorate each tree node with orientation (r,q)
  and then model dependencies thru scale

35
Multiscale Geometry Model (MGM)

• Decorate each tree node with orientation (r,q)
  and then model dependencies thru scale
• Insight Smooth curve Geometric innovations
  small at fine scales
• Model Favor small innovations over large
  innovations (statistically)

36
Multiscale Geometry Model (MGM)

• Wavelet-like geometry model
  coarse-to-fine prediction
• model parent-to-child transitions of orientations

small innovations
large
37
MGM

• Wavelet-like geometry model
  coarse-to-fine prediction
• model parent-to-child transitions of orientations
• Markov-1 statistical model
• state (r,q) orientation of wedgelet
• parent-to-child state transition matrix

38
MGM

• Markov-1 statistical model
• Joint wedgelet Markov probability model
• Complexity Shannon codelength
  number of bits to encode

39
MGM and Edge Smoothness
smooth simple
rough complex
40
MGM Inference

• Find representation / prune tree to balance the
  fidelity vs. complexity trade-off
• Efficient solution via dynamic programming

41
Wedgelet Coding of Cartoon Images

• Choosing wedgelets rate-distortion optimization

Shannon code length
to encode
42
Predictive Wedgelet Coding
Optimal rate-distortion performance
compared to
for leaf-only encoding
43
Proof Start with leaf-encoding Consider a
wedgelet at scale j Block size is 2-j x 2-j
O(2-2j)
2-j
C2 curve contained in strip with width length2
44
Choose discrete wedgelet dictionary with tick
marks of spacing O(2-2j)
O(2-2j)
2-j
45
Discrete Wedgelet Dictionary
46
A discrete wedgelet exists that deviates from
curve by only O(2-2j) L2 distortion on this
block O(2-3j)
O(2-2j)
2-j
Since curve is C2, there are O(2j) such
blocks Total distortion DO(2-2j)
47
Number of bits (bitrate) to encode tick marks
O(j) bits per block thus, total rate R
O(j2j)
O(2-2j)
2-j

Combining
for leaf-only encoding
48
Can reduce bitrate by predicting wedgelets down
through scale (MGM) consider scale j1
O(2-2(j1))
2-(j1)
2-(j1)
49
Finer scale wedgelets within O(2-2(j1)) of
curve Coarse scale wedgelet within O(2-2j) of
curve
O(2-2(j1))
2-(j1)
2-(j1)
Given the parent, there are only O(1) possible
wedgelets on each finer block. Thus, rate is O(1)
bits per block, or RO(2j) bits total. This gives
the optimal D(R)R-2.
50
Summary Wedgelets

• Adaptive dyadic partition
• Fit to data using fidelity/complexity
  optimization
• Improve using MGM statistical model for
  parent-to-child wedgelet orientation
  correlations
• MGM parent-to-child modeling akin to wavelets
  for wedgelets
• Optimal approximation and R/D rates
• Dynamic programming as a useful tool
• Only suited for cartoon images

51
n-D Surflet Representation VC,MW,DB,RB

• Generalize wedgelet concept to
• arbitrary dimension n
• arbitrary (n-1)-D manifold smoothness k
• requires gtlinear elements

52
Joint Texture/Geometry Modeling

• Dictionary D wavelets U wedgelets
• Representation tradeoff
  texture vs. geometry
• Test case approximation / compression