Shadow removal algorithms

1
Shadow removal algorithms

• Shadow removal seminar
• Pavel Knur

2
Deriving intrinsic images from image sequences

• Yair Weiss
• July 2001.

3
History

• intrinsic images by Barrow and Tenenbaum 1978

4
Constraints

• Fixed viewpoint
• Works only for static objects
• Cast shadows

5
Classic ill-posed problem

• Denote
• the input image
• the reflectance image
• the illumination image
• Number of Unknowns is twice the number of
  equations.

6
The problem

• Given a sequence of T images
• in which reflectance is constant over
• the time and only the illumination
• changes can we solve for a single
• reflectance image and T illumination
• images
• Still completely ill-posed at every pixel there
  are T equations and T1 unknowns.

7
Maximum-likelihood estimation

• Log domain

8
Assumptions

• When derivative filters are applied to natural
  images the filter outputs tend to be sparse.

9
Laplacian distribution

• Can be well fit by laplacian distribution

10
Claim 1

• Denote
• N filters
• Filter outputs
• Filtered reflectance image
• ML estimation of filtered reflectance image
• is given by

11
Estimated reflectance function

• Recover ML estimation of r
• is reversed filter of

12
ML estimation algorithm
13
ML estimation algorithm cont.

• Ones we have estimated

14
Claim 2

• What if does not have exactly a
  Laplasian distribution
• Let
• Then estimated filtered reflectance are within
  with probability at least

15
Claim 2 - proof

• If more than 50 of the samples of
• are within of some value then by definition
  of median the median must be within of that
  value.

16
Example 1

• Einstein image is translated diagonally
• 4 pixels per frame

17
Example 2

• 64 images with variable lighting from Yale Face
  Database

18
Illumination Normalization with Time-Dependent
Intrinsic Images for Video Surveillance

• Y.MatsushitaK.NishitoK.Ikeuchi
• Oct. 2004

19
Illumination Normalization algorithm

• Preprocessing stage for robust video
  surveillance.
• Causes
• Illumination conditions
• Weather conditions
• Large buildings and trees
• Goal
• To normalize the input image sequence in terms
  of incident lighting.

20
Constraints

• Fixed viewpoint
• Works only for static objects
• Cast shadows

21
Background images
Input images

• Remove moving objects from the input image
  sequence

Off-line
Background images
22
Estimation of Intrinsic Images
Input images

• Denote
• input image
• time-varying reflectance image
• time-varying illumination image
• reflectance image estimated by ML
• illumination image estimated by ML
• Filters
• Log domain

Off-line
Background images
Estimation of Intrinsic Images
23
Estimation of Intrinsic Images cont.
Input images

• In Weisss original work
• The goal is to find estimation of and

Off-line
Background images
Estimation of Intrinsic Images
24
Estimation of Intrinsic Images cont.
Input images

• Basic idea
• Estimate time-varying reflectance components by
  canceling the scene texture from initial
  illumination images
• Define

Off-line
Background images
Estimation of Intrinsic Images
25
Estimation of Intrinsic Images cont.
Input images

• Finally
• Where
• is reversed filter of

Off-line
Background images
Estimation of Intrinsic Images
26
Shadow Removal
Input images

• Denote
• - background image
• - illuminance-invariant image

Off-line
Background images
Estimation of Intrinsic Images
27
Illumination Eigenspace
Input images

• PCA Principle component analysis
• Basic components -

Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
28
Illumination Eigenspace cont.
Input images

• Average is
• P is MxN matrix where
• N number of pixels in illumination image
• M number of illumination images
• Covariance matrix Q of P is

Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
29
Direct Estimation of Illumination Images
Input images

• Pseudoillumination image
• Direct Estimation is
• Where
• F is a projection function onto the js
  eigenvector
• -

Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
30
Direct Estimation of Illumination Images
Input images

• Results

Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
31
Shadow interpolation
Input images

• probability density function
• cumulative probability function
• shadowed area
• lit area
• mean
• optimum threshold value

Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
Shadow Interpolation
32
The whole algorithm
Input images
Off-line
Background images
Estimation of Intrinsic Images
Illumination Eigenspace
Shadow Interpolation
Illumination Images
/
Normalization
33
Example
34

• Questions

35
References

• 1 Y.WeissDeriving Intrinsic Images from Image
  Sequences Proc. Ninth IEEE Intl Conf. Computer
  Vision pp. 68-75 July 2001.
• 2 Y.MatsushitaK.NishitoK.IkeuchiIllumination
  Normalization with Time-Dependent Intrinsic
  Images for Video SurveillanceOct. 2004.