A Markov Random Field Framework for Finding Shadows in a Single Colour Image

1
A Markov Random Field Framework for Finding
Shadows in a Single Colour Image

• Cheng Lu and Mark S. Drew
• School of Computing Science, Simon Fraser
  University, Vancouver (CANADA)
  clu,mark_at_cs.sfu.ca

2
Objective finding shadows
Many computer vision algorithms, such as
segmentation, tracking, and stereo registration,
are confounded by shadows.
Finding shadows
3
Shadows stem from what illumination effects?

• Changes of illuminant in both intensity and
  colour

Intensity sharp intensity changes
Colour shadows exist in the chromaticity image
Region Lit by Sunlight and Sky-light
Region Lit by Sky-light only
4
Colour of illuminants

• Wiens approximation of Planckian illuminants
• How good is this approximation?

2500 Kelvin
10000 Kelvin
5500 Kelvin
5
Invariant Image Concept
Finlayson et al.,ECCV2002
n
ai
Planckian Lighting
x
The responses
k R, G, B
Lambertian Surface
For narrow-band Sensors
Shading and intensity term
6
Band-ratio chromaticity
Let us define a set of 2D band-ratio
chromaticities
p is one of the channels, (Green, say) or
could use Geometric Mean
Perspective projection onto G1
7
Band-ratios remove shading and intensity
Lets take logs
Shading and intensity are gone.
Gives a straight line
8
Calibration find illuminant direction
Illuminant direction
Invariant direction
Macbeth ColorChecker 24 patches
Log-ratio chromaticities for 6 surfaces under 14
different Planckian illuminants, HP912 camera
9
A real image containing shadows

• Two lights
• Shadows lit by sunlight and sky-light
• Non-shadows lit by sky-light

The red line refers to the changes of
illuminants same surface lit by two different
lights
10
Illuminant discontinuity

• Illuminant discontinuity pair
• Two neighbouring pixels of a single surface,
  under two different lights

Illuminant discontinuity pair
11
Illuminant discontinuity measure

• Using the means of two neighboring blocks of
  pixels
• better than using two neighbouring pixels because
  of noise and diffuse shadow edges.
• Illuminant discontinuity angle
• Cos of the two vectors

12
Finding Shadows

• Label image pixels with label l shadow,
  nonshadow
• Model this labelling problem using Markov Random
  Field
• The label of a pixel depends only on its
  neighbours

First order neighbors 
13
Markov Random Field

• l is a Markov Random Field
• l follows a Gibbs distribution Znormalizing
  constant, and
• U(l) is an energy function defined with respect
  to neighbours
• labelling minimizing energy U(l)

14
Energy function
Roughly,
In full,
15
Implementation

• Gibbs Sampler can be used to minimize the energy
  optimization technique.
• Texture and noise may confuse the discontinuity
  measure, so the Mean Shift method is used to
  filter (segment) the image first.

16
Experiments