Graph Cut Algorithms for Binocular Stereo with Occlusions

1
Graph Cut Algorithms for Binocular Stereo with
Occlusions
Vladimir Kolmogorov, Ramin Zabih
2
Overview

• Traditional Stereo Methods
• Energy Minimization via Graph Cuts
• Stereo with Occlusions
• Voxel Labeling Algorithm
• Pixel Labeling Algorithm
• Results and Conclusions

3
Traditional Stereo Methods
Traditional Stereo Problem

• pixel correspondences ? labeling (disparity)

4
Traditional Stereo Methods Disparity
disparity
depth
? disparity depth
ground truth disparity
5
Traditional Stereo MethodsBinocular Stereo

• goal is to compute pixels correspondences
• traditional stereo problem ? pixel labeling
  problem
• advantage can be solved by graph cuts
• problem is formulated as energy term
• new goal find the minimizing labeling

6
Traditional Stereo MethodsEnergy Function
find labeling
that minimizes
cost for assigning labels
smoothness term
7
Traditional Stereo MethodsEnergy Function

• data cost gives penalty for different
  intensities

• smoothness term gives penalty for
  discontinuities (Potts model)

other models absolute distance quadratic
8
Energy Minimization via Graph Cuts
Max-flow / Min-Cut (Ford and Fulkerson Algorithm,
Push-Relabel Method)
9
Energy Minimization via Graph Cuts

• convex V vs. metric / semimetric
• a-ß-swap move
• a-expansion move assigning label a to an
  arbitrary set of pixels

10
Stereo with Occlusions
11
Stereo with Occlusions

• treat input symmetrically
• scene elements only visible in single view
• physically correct scenes ?
  geometric constraints ? occlusions ?
  physically possible labelings
• introduce constraints in the problem formulation
• graph cuts perform unconstrained energy
  minimization

12
Voxel Labeling Algorithm

• discrete scene of voxels
• voxel v is active when visible from both cameras
• uniqueness constraint 11 correspondence of
  pixels

13
Voxel Labeling AlgorithmEnergy Function
smoothness term (Potts model)
matching penalty (only active voxels)
occlusion penalty
set of occluded pixels
14
Pixel Labeling AlgorithmEnergy Function

• like traditional stereo but for both images e.g.
  Potts model

active ?
15
Minimizing the Energy
(0valid, else 8) uniqueness

• convert constrained into unconstrained
  minimization problem
• write as sum over pairs
• form of energy function standard stereo problem
• minimization with a-expansion algorithm
• modified definition of a-expansion move for voxel
  labeling

16
Results and Conclusions
ground truth
Tsukuba ref. image
17
Results and Conclusions

• efficient energy minimization polynominal
  time instead of exponential time
• traditional stereo algorithm is faster
• pixel labeling better than voxel labeling
• prohibits holes in the scene
• allows to use other effective smoothness terms
• algorithms can be extended for multiple cameras

18
Multi-view Stereo via Volumetric Graph Cuts
19
Recent Work

• Graph-cut-based stereo matching using image
  segmentation with symmetrical treatment of
  occlusions, 2006 TUW

20
Questions?
21
References

• M. Bleyer, M. Gelautz, Graph-cut-based stereo
  matching using image segmentation with
  symmetrical treatment of occlusions, 2007
• Y. Boykov, O. Veksler, R. Zabih, Fast
  Approximate Energy Minimization via Graph Cuts,
  2001
• V. Kolmogorov, R. Zabih, Graph Cut Algorithms
  for Binocular Stereo with Occlusions, 2005
• V. Kolmogorov, R. Zabih, What energy functions
  can be minimized via graph cuts, 2004
• V. Kolmogorov, R. Zabih, Generalized
  multi-camera scene reconstruction using graph
  cuts, July 2003
• V. Kolmogorov, R. Zabih, Multi-camera Scene
  Reconstruction via Graph Cuts, 2002
• S. Seits, C. Dyer, Photorealistic Scene
  Reconstruction by Voxel Coloring, 1997
• R.Szeliski, R. Zabih, An Experimental Comparison
  of Stereo Algorithms, 1999