Jitter Camera: High Resolution Video from a Low Resolution Detector

1
Jitter Camera High Resolution Video from a Low
Resolution Detector

• Moshe Ben-Ezra, Assaf Zomet and Shree K. Nayar
• IEEE CVPR Conference
• June 2004, Washington DC, USA

2
Video Resolution
3
Fundamental Resolution Tradeoff
3
Temporal resolution (fps)
30
130
Spatial resolution (pixels)
4
Super-Resolution
Super-Resolution
Sequence taken by a moving camera
High-Resolution computed image
5
What is Super Resolution?
6
What is Super Resolution
S11
S12
S S11 S12 S21 S22
S21
S22
7
Super Resolution
y (D G)x z
Blurring Op.
Noise
All Sampled Images
Decimation
Hi Res. Image
8
Motion Blur Hurts Us Again!
9
Capture Images without Motion Blur
10
Effect of Motion Blur on Super-Resolution
11
Quantifying The Affect of Motion Blur

• Empirical tests RMS error.
• Volume of Solutions (Linear Model)

Baker and Kanade
12
How Bad is Motion Blur for Super-Resolution?
Space of Super-Resolution Solutions
RMS Error After Super-Resolution
0 1 2 3 4
5
Motion blur in pixels
Motion blur in pixels
13
Avoid Motion Blur using Jitter Sampling
Conventional Sampling
14
The Jitter Camera
Lens
Detector
Micro-actuator
15
The Jitter Camera
Lens
Detector
Micro-actuator
Detector is a light weight device!
Jitter is instantaneous and synchronous
16
Computer Controlled X Micro-Actuator
Computer Controlled Y Micro-Actuator
Lens
Board Camera
17
(No Transcript)
18
Jitter Mechanism Accuracy
Y Pixels
X Pixels
Desired locations.
Actual locations.
19
Result Resolution Chart
Super-Resolution Image
Four Images from the Jitter Camera
20
De-Mosaic
Artifacts around edges
21
Result Color DeMosaicing and Super-Resolution
1 (out of 4) Jitter camera Image
22
Jitter Video (Stabilized)
How can we handle dynamic scenes?
23
Adaptive Super-Resolution for Dynamic Scenes
24
(No Transcript)
25
Adaptive Super-Resolution Algorithm
I-3
I-2
I-1
I
I1
I2
I3

• Estimate the aliasing error ? (stdv) for each
  block Ik in I.
• Compute robust block matching between all pairs
  II?1,2,3. Use ? as a scale factor for an
  M-Estimator error function.
• For each block Ik try to find 3 matching blocks
  I?xk, s.t.
• SSD(Ik, I?xk)-0.5 lt 3?
• I?xk are temporally closest to Ik (smallest x)
• Apply super-resolution to the selected blocks.
• The algorithm degrades gradually from 4-frames
  super-resolution to a single frame interpolation
  and deblurring.

26
Scale Estimate
Mean 6.4, Stdv 14
Mean 7.5, Stdv 16
Mean 8.6, Stdv 17
Low Res - Hi-Res Aliasing Error (Simulated)
Mean 10.5, Stdv 27
Mean 15.2, Stdv 30
Mean 17.7, Stdv 33
Low Res 2nd derivative (Simulated)
27
(No Transcript)