Fast Marching Method for generic Shape From Shading

1
Fast Marching Method for generic Shape From
Shading
E. Prados S. Soatto
RFIA 2006 January 2006,  Tours, France
2
Explicit generic SFS equation

• Hypotheses
• Lambertian Reflectance,
• Lighting
• Single punctual light source,
• Located at the infinity or optical center,
• Possibility to take into account the attenuation
  of the light due to the distance,
• Orthographic or perspective camera .

3
Explicit generic SFS equation
Particular case of the generic PDE

F(u)
PradosPhD'04
4
Explicit generic SFS equation
Particular case of the generic PDE

F(u)

• Particular characteristics
• Dependency in u (no only in ?u).
• Solution not necessarily increasing along the
• characteristic curves...
• Current Fast Marching Methods do not apply.

5
Fast Marching Method (FMM)

• One-pass method for solving numerically PDEs
• ? iterative methods
• No threshold as stopping criterion
• Optimal number of updates ? Low computation time
• Based on front propagation.

Iterative method
FMM
Iterative method
6
Fast Marching Method (FMM)

• One-pass method for solving numerically PDEs
• ? iterative methods
• No threshold as stopping criterion
• Optimal number of updates ? Low computation time
• Based on front propagation.

Iterative method
FMM
Iterative method
7
Fast Marching Method (FMM)

• One-pass method for solving numerically PDEs
• ? iterative methods
• No threshold as stopping criterion
• Optimal number of updates ? Low computation time
• Based on front propagation.

Iterative method
FMM
Iterative method
8
Fast Marching Method (FMM)

• One-pass method for solving numerically PDEs
• ? iterative methods
• No threshold as stopping criterion
• Optimal number of updates ? Low computation time
• Based on front propagation.

Iterative method
FMM
Iterative method
9
Fast Marching Method (FMM)

• One-pass method for solving numerically PDEs
• ? iterative methods
• No threshold as stopping criterion
• Optimal number of updates ? Low computation time
• Based on front propagation.
• Many applications
• Path planning Kimmel-Sethian01
• Geometric optics Wenwang03
• Image processing and computer vision
  L.Cohen05
• Extensive list, see Sethian99.

Iterative method
FMM
Iterative method
10
Fast Marching Method (FMM)
(x,y1)

• Basic method Sethian99, Dijkstra59
• ?u g(x), (Eikonal equation)
• 4-neighborhood scheme.
• Most recent extension OUM Sethian-Vladimirsky0
  3
• supa f(x,a) a.?u -1 0, (2)
• Scheme requiring very large neighborhood
• (size depending on the
  anisotropy)
• Our extension (new algorithm)
• ? F(u) H(x,?u) 0, (3)
• where F is strictly increasing and H is
  convex
• 4-neighborhood scheme.

(x,y)
(x1,y)
(x-1,y)
(x,y-1)
11
Our New Extension of the FMM

1. A New Numerical Scheme
2. A New Causality

12
A New Numerical Scheme

• Preliminary step
• Legendre Tranform equation as a supremum
• ? F(u) H(x,?u) 0
• ? F(u) sup f(x,a).?u(x) - l(x,a) 0

Cost function
13
A New Numerical Scheme

• We approximate
• And we choose the simplex, i.e. the si such that
  the scheme is increasing with respect to t
  (represent u)
• We then choose si si(x,a) sign fi(x,a) ,
• In other words, we choose the simplex which
  contains the optimal trajectory (see next
  section).

t ?? u(x) si ? 1, -1
? F(t)
14
A New Numerical Scheme

• Our scheme is provably consistent and monotonic
  
• Only direct neighborhoods
• in SFS with regular mesh 4-neighborhood.
• Equation depends on u

15
A New Causality and Reinterpretation

• Key point Distinction of the causality and of
  the simultaneous integration.
• Causality Theoretic propagation of the
  information
• ensures that we compute approximation of the
  viscosity solution !
• The information propagation follows specific
  curves
• the optimal trajectories
• The solution can be computed by direct
  integration along these curves
• Trying to compute the solution curve after curve
  is numerically unstable
• ? Simultaneous integration along all the curves

16

• Simultaneous integration ensures the
  numerical stability !
• ? Propagation front
• How to choose the propagation front?
• Many propagation fronts following the optimal
  trajectories can be designed!

17

• Simultaneous integration ensures the
  numerical stability !
• ? Propagation front
• How to choose the propagation front?
• Many propagation fronts following the optimal
  trajectories can be designed!

18

• Simultaneous integration ensures the
  numerical stability !
• ? Propagation front
• How to choose the propagation front?
• Many propagation fronts following the optimal
  trajectories can be designed!

19

• Simultaneous integration ensures the
  numerical stability !
• ? Propagation front
• How to choose the propagation front?
• Many propagation fronts following the optimal
  trajectories can be designed!
• How to define the propagation front? idea
  introduction of a cost C such that level sets
  of C correspond with propagated front.

20

• Simultaneous integration

Optimal trajectory
? All previous SFS methods choose
C u ( u is the solution)
inconsistent with the causality when
the solution does not increase along the
optimal trajectories ! ? We show how to define
an appropriate cost C - which is always
consistent with optimal trajectories, -
which allows to define and compute simply and
practically the update order ? This cost C is
based on the notion of subsolution ?
Level sets of u
21
Fast Marching Method (FMM)
Choice of the pixel based on causality
Values computed using the new scheme
Domain where we know the solution
22
Algorithm
A points acceptés
F points éloignés
C points considérés
23
A Practical and Detailed Example

• The classical Rouy/Tourin equation
• Associated Modeling
• Lambertian and homogeneous reflectance,
• Single oblique and far light source,
• orthographic projection.
• Resulting PDE
• where I(x) is the image.
• Associated cost function
• have arbitrary signs
• Solution does not increase along the
  characteristic curves,
• All previous Fast Marching Methods do not
  directly apply !
• Associated subsolution

24
Focus on the improvement due to the new Causality
Reconstruction with the new and correct causality
Reconstruction with the classical causality
original image
oblique view
oblique view
groundtruth
Computed solution
groundtruth
profile view
profile view
25

• SFS with attenuation of light due to distance
• Associated Modeling
• Lambertian and homogeneous reflectance,
• Single proximal light source (optical center),
• perspective projection.
• Resulting PDE I(x) image and f focal
  length.
• and
• All previous Fast Marching Methods do not
  directly apply

image
reconstruction
26
References
References

• Kimmel-Sethian01 R. Kimmel and J.A. Sethian.
  Optimal algorithm for shape from shading and path
  planning. JMIV, 14(2)237244, May 2001.
• L.Cohen05 L. Cohen. Minimal paths and fast
  marching methods for image analysis. In
  Mathematical Models in Computer Vision The
  Handbook, Springer, 2005.
• Sethian99 J.A. Sethian. Level Set Methods and
  Fast Marching Methods. Cambridge University
  Press,1999.
• Sethian-Vladimirsky03 J.A. Sethian and A.
  Vladimirsky. Ordered upwind methods for
  HamiltonJacobi equationsTheory and algorithms.
  SIAM J. on Num. Ana. 41(1), 2003
• Prados04 E. Prados. Application of the theory
  of the viscosity solutions to the Shape From
  Shading problem. PhD thesis, 2004.