Title: Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos
1Spatio-Temporal Frequency Analysis for Removing
Rain and Snow from Videos
Peter Barnum Takeo Kanade Srinivasa Narasimhan
Carnegie Mellon University
June 16, 2007
2Bad weather in outdoor videos
Rain
Snow
3Previous Work
Image-based blurring
Camera-based blurring
Spikes due to rain
Pixel Intensity
Garg and Nayar ICCV 05
Streak detection
Time
Hase et al. ICIP 98 Starik and Werman IWTAS
03 Zhang et al. ICME 06
Garg and Nayar CVPR 04
4Groups of streaks
5Imaging a falling particle
Gaussians
Snowflakes
Raindrops
Breadth
6Gaussian model of streak appearance
Camera parameters are constant
A Gaussian
A blurred Gaussian streak
Including streak orientation (just a coordinate
space rotation)
7Where are the streaks?
8Fourier transform of the streaks
9Building a complete model
For a given precipitation intensity
For all common drop sizes
Blurred Gaussian
For all depths that are in-focus
10Model accuracy
Original image
2D Fourier Transform
Model
Model with 50 randomly set to zero
11Finding the precipitation rate
Rain and snow have two useful properties
Mailbox
Building
100
0
Snow
12Frame-to-frame differences
t1
t3
t2
w-1
w1
w0
13Frame-to-frame differences
t1
t3
t2
w-1
w1
w0
14Finding the precipitation rate
For most objects
But for rain and snow
Because of these properties
15Estimating the model parameters
Precipitation rate
Streak orientation
The precipitation rate is approximately
Estimating the streak orientation requires a
spatial consistent estimate
The orientation is found by
16Frequency space examples
Original image
2D FT
Model
17Computing per-frequency estimates
At a given frequency
18Computing per-pixel estimates
19Refining the single frame estimate
Exactly the same model, constant in w
20Computing per-pixel estimates
21(No Transcript)
22(No Transcript)
23(No Transcript)
24Conclusions and future work
A global frequency method for rain and snow
removal
Refining global estimates with local features
Into each life some rain must fall. -Henry
Wadsworth Longfellow
25Extra slides
26Imaging a falling particle