Stereo Matching Using Dynamic Programming

1
Stereo Matching Using Dynamic Programming

• Jim Rehg
• CS 4495/7495 Computer Vision
• Lecture 4
• Mon Sept 2, 2002

2
Correspondence

• It is fundamentally ambiguous, even with stereo
  constraints

Ordering constraint
and its failure
3
Search Over Correspondences
Left scanline
Right scanline

• Three cases
• Sequential cost of match
• Occluded cost of no match
• Disoccluded cost of no match

4
Stereo Matching with Dynamic Programming
Left scanline
Start

• Dynamic programming yields the optimal path
  through grid. This is the best set of matches
  that satisfy the ordering constraint

Dis-occluded Pixels
Right scanline
End
5
Dynamic Programming
1
1
1
2
2
2
3
3
3
Principle of Optimality for an n-stage assignment
problem
6
Dynamic Programming
1
1
1
2
2
2
3
3
3
Principle of Optimality for an n-stage assignment
problem
7
Dynamic Programming
1
1
1
2
2
2
3
3
3
Principle of Optimality for an n-stage assignment
problem
8
Dynamic Programming
1
1
1
2
2
2
3
3
3
Principle of Optimality for an n-stage assignment
problem
9
Dynamic Programming
1
1
1
2
2
2
3
3
3
Principle of Optimality for an n-stage assignment
problem
10
Dynamic Programming
1
1
1
2
2
2
3
3
3
Back-chaining recovers the optimal path and its
cost
11
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
12
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
13
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
14
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
15
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
16
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
17
Stereo Matching with Dynamic Programming
Left scanline

• Scan across grid computing optimal cost for
  each node given its upper-left neighbors.Backtrac
  k from the terminal to get the optimal path.

Dis-occluded Pixels
Right scanline
Terminal
18
Computing Correspondence

• Another approach is to match edges rather than
  windows of pixels
• Which method is better?
• Edges tend to fail in dense texture (outdoors)
• Correlation tends to fail in smooth featureless
  areas

19
Computing Correspondences

• Both methods fail for smooth surfaces
• There is currently no good solution to the
  correspondence problem