Title: Image Destabilization: Programmable Defocus using Lens and Sensor Motion
1Image DestabilizationProgrammable Defocus using
Lens and Sensor Motion
MIT Media Lab Camera Culture
- Ankit Mohan, Douglas Lanman,Shinsaku Hiura,
Ramesh RaskarMIT Media Lab
2Defocus Blur
3Image Destabilization
Camera
Lens
Sensor
Static Scene
4Image Destabilization
Camera
Static Scene
Lens Motion
Sensor Motion
5Related Work
f8
f/4
f/2
extrapolated aperture
f/1
Hasinoff and Kutulakos 2007
Vaish et al. 2004
Hiura et al. 2009
6Laminography
Motion direction
X-Ray Source
Plane of focus
X-Ray Sensor
Motion direction
Related technique Time Delay and Integration
(TDI)
7Lens based Focusing
Lens
Sensor
B
A
A
B
8Lens based Focusing
Lens
Sensor
B
A
A
B
9Smaller aperture ? Smaller defocus blur
Lens
Sensor
B
A
A
B
10Pinhole All In-Focus
Pinhole
Sensor
B
A
A
B
11Shifting Pinhole
Pinhole
Sensor
vp
A
B
B
A
12Shifting Pinhole
Pinhole
Sensor
vp
A
B
B
A
13Shifting Pinhole
Pinhole
Sensor
vp
B
A
A
B
14Shifting Pinhole
Pinhole
Sensor
vp
B
A
A
B
15Shifting Pinhole
Pinhole
Sensor
vp
B
A
tp
A
B
da
db
ds
16Shifting Pinhole and Sensor
Pinhole
Sensor
vp
vs
A
B
B
A
da
db
ds
Focus Here
17Shifting Pinhole and Sensor
Pinhole
Sensor
vp
vs
B
A
A
B
da
db
ds
Focus Here
18Shifting Pinhole and Sensor
Pinhole
Sensor
B
vp
vs
A
A
B
da
db
ds
Focus Here
19Shifting Pinhole and Sensor
Pinhole
Sensor
B
vp
vs
A
A
B
da
db
ds
Focus Here
20A Lens in Time!
Lens Equation
Virtual Focal Length
Virtual F-Number
Analogous to shift and sum basedLight field
re-focusing.
21Our Prototype
22Adjusting the Focus Plane
all-in-focus pinhole image
23Adjusting the Focus Plane
focused in the front using destabilization (10
second exposure)
24Adjusting the Focus Plane
focused in the middle using destabilization (5
second exposure)
25Adjusting the Focus Plane
focused in the back using destabilization (10
second exposure)
26Adjusting the Virtual Aperture
focused in the middle using destabilization (5mm
pinhole translation)
27Adjusting the Virtual Aperture
focused in the middle using destabilization (30mm
pinhole translation)
28Shifting Lens and Sensor Defocus
Defocus Exaggeration
- Physical vs. synthetic focus
- Similar physical and synthetic focus
aperture
sensor
29Defocus Exaggeration
static lens with an f/2.8 aperture
30Defocus Exaggeration
destabilization simulates a reduced f-number
31Defocus Invariance
- Differing physical and synthetic focus
virtual focus
aperture
sensor
32Defocus Invariance
- Related work
- Nagahara et al. 2008
- Cathey and Dowski 1995
- Levin et al. 2008
- PSF not depth invariant
- only size is depth invariant
- Gaussian special case
- depth invariant PSF
- inversion is ill-conditioned
33Defocus Invariance
depth-invariant blur size (horizontal slit
destabilization)
34Defocus Invariance
Richardson-Lucy deconvolution result
35Tilted Sensor
aperture plane
sensor plane
focus plane
36Tilted Sensor
aperture plane
sensor plane
focus plane
37Tilted Sensor
aperture plane
sensor plane
focus plane
38Tuning the PSF
real focus
aperture
sensor
pinhole image (static f/22 aperture)
39Tuning the PSF
real focus
aperture
sensor
large aperture image (static f/2.8 aperture)
40Tuning the PSF
virtual focus
aperture
sensor
destabilized image using a pinhole (translated
f/22 aperture)
41Tuning the PSF
real focus
virtual focus
aperture
sensor
destabilized image using a large aperture
(translated f/2.8 aperture)
42Tuning the PSF
real focus
virtual focus
aperture
sensor
simulated aspheric lens using a vertical slit
aperture and destabilization
43Extension to 2D Displacements
44Large apertures with tiny lenses?
- Limitations
- Coordinated mechanical movement required
- Diffraction (due to small aperture) cannot be
eliminated - Zhang and Levoy, tomorrow
- Our group augmented LF for wave
analysis - Scene motion during exposure
- Benefits
- No time or light inefficiency wrt cheap cameras
- Exploits unused area around the lens
- Compact design
- With near-pinhole apertures (mobile phones) many
possibilities
45Acknowledgements
Grace Woo MIT CSAIL
Quinn Smithwick MIT Media Lab
Gabriel Taubin Brown University
Jaewon Kim MIT Media Lab
MIT Media Lab Camera Culture
46SLRs with tiny lenses?
- Analysis of space of relative lens/sensor
displacement - Destabilization as virtual focusing mechanism
- Shallower depth of field than physical aperture
- Depth-independent defocus blur size