INTEGRATION OF INTENSITY EDGE INFORMATION INTO THE REACTION-DIFFUSION STEREO ALGORITHM

1
INTEGRATION OF INTENSITY EDGE INFORMATION
INTOTHE REACTION-DIFFUSION STEREO ALGORITHM

• Atsushi Nomura1) Makoto Ichikawa2)
• Koichi Okada1) Hidetoshi Miike1)
• 1)Yamaguchi University, Japan
• 2)Chiba University, Japan

2
In this talk,

• We integrate intensity edge information into a
  reaction-diffusion stereo algorithm.
• Depth discontinuity gt error of disparity
  detection.
• Anisotropic diffusion.

3
Outline of this talk

• Stereo vision, stereo algorithm related topics
• Reaction-diffusion equations reaction-diffusion
  algorithm
• Proposed stereo algorithm integrating intensity
  edge information
• Experimental results
• Conclusion future work

4
Binocular Stereo Vision
Object
Object
Optical axis
Object
Depth
Disparity dxL-xR
Cross-correlation
IL(x,y)
IR(x,y)
. . .
Object
(xL,y)
(xR,y)
Focul length
Left eye
Right eye
stereo correspondence problem gt segmentation
problem
dN-1
. . .
d0
for possible disparity levels
5
Previous Stereo Algorithms

• Marr Poggio, Proc. Roy. Soc. Lond., 1979
• original cooperative algorithm
• continuity uniqueness constraints
• Zitnick Kanade, IEEE-PAMI, 2000
• modern cooperative algorithm occlusion
  detection
• Sun et al., IEEE-PAMI, 2003
• belief-propagation algorithm
• Deng et al., IEEE-PAMI, 2007
• graph-cuts algorithm

6
Previous Edge Detection Algorithm

• Marr Hildreth, Proc. Roy. Soc. Lond., 1980
• LoG filter DOG filter
• Canny, IEEE-PAMI, 1986
• Standard algorithm
• Heath et al., IEEE-PAMI, 1997
• Review of edge detection algorithms
• Standard images

7
Diffusion Equation PDE Approach inImage
Processing Computer Vision Research

• Koenderink, Biol. Cybern., 1984
• Diffusion equation Gaussian filter
• Perona Malik, IEEE-PAMI, 1990
• Anisotropic diffusion
• Black et al., IEEE-IP, 1998
• Anisotropic diffusion
• Mrázek Navara, IJCV, 2003
• Stopping time for non-linear diffusion filtering
• Galié et al., J. Math. Imaging Vis., 2008
• Image compression using anisotropic diffusion

8
Integration of Visual Cuesin Human Vision System

• Landy et al., Vis. Res., 1995
• Fusion of multiple visual cues in depth
  perception
• Linear fusion vs. weak fusion
• Ichikawa et al., Vis. Res., 2003
• Non-linear integration model of disparity other
  visual cues.

9
Reaction-Diffusion Algorithm

• Adamatzky et al., Reaction-Diffusion Computers,
  2005
• proposing novel computer architecture, by
  utilizing reaction-diffusion equations.
• Suzuki et al., Trans. IEICE, 2005
• image restoration LSI implementation
• Asai et al., Int. J. Bifurcation Chaos, 2005
• Voronoi diagram LSI implementation
• Solving partial differential equations needs much
  computation power. Hardware (LSI) implementation
  has been introduced.

10
Reaction-Diffusion Equations

• A set of partial differential equations having
  diffusion terms and reaction terms.

u(x,y,t), v(x,y,t) variables Du, Dv diffusion
coefficients f(u,v), g(u,v) reaction terms

• FitzHugh-Nagumo type reaction terms

Constants 0lteltlt1 a, b
FitzHugh, Biophysical J., 1961 Nagumo et al.,
Proc. IRE, 1962
11
Ordinary Differential Equation (ODE) System
(DuDv0)

• Mono-stable system bi-stable system

Mono-stable
a0.25,b1.0 e1/100
a threshold value
Bi-stable
a0.25,b10.0 e1/100
12
Reaction-Diffusion Equations inOne-Dimensional
Case

• FitzHugh-Nagumo equations bi-stable system

Parameter setting Du1.0, Dv3.0 a0.05,
b10.0, e1/100
Propagation speed depends on the diffusion
coefficients of Du and Dv.
13
Discrete System of Reaction-Diffusion Equations
in One-Dimensional Space

• FitzHugh-Nagumo equations (discrete system)

DultltDv (Du1.0, Dv5.0), D1/4, a0.25, e1/1000,
i index of spatial position
Bi-stable system (b10.0)
Mono-stable system (b1.0)
i
i
Initial conditions ui(t) step function and
vi(t0)0.0 (uniform)
14
Reaction-Diffusion Stereo Algorithm

• Nomura et al., Mach. Vis. Appl., (in press)

The diffusion term Du?2un drives the propagation
of region of the disparity level dn.
m constant N total number of possible
disparity levels C similarity measure dn
disparity level
Disparity map
15
Reaction-Diffusion Stereo Algorithm (cont.)
The uniqueness constraint is realized by,
This term realizes the uniqueness
constraint. Other set having umax increases this
threshold level.
Wi inhibition area
16
Flow Chart of Reaction-Diffusion Stereo Algorithm
edge detection result ve
integration
edge detection result ue
C(x,y,d0)
u0(x,y,t)
v0(x,y,t)
self-inhibition due to DultltDv
Left image IL(x,y)
dn (pixel)
. . .
. . .
. . .
Mutual inhibition through f(?,?,?)
Right image IR(x,y)
C(x,y,dn)
un(x,y,t)
vn(x,y,t)
M(x,y,t)
. . .
. . .
. . .
Stereo disparity map
C(x,y,dN-1)
uN-1(x,y,t)
vN-1(x,y,t)
Stereo images
Correlation maps
Reaction-diffusion systems
Computation of similarity measure at each
disparity level dn
17
Edge Detection Algorithm withReaction-Diffusion
Equations

• Nomura et al., J. Phys. Soc. Jpn., 2003
• Edge detection Segmentation
• Discrete version of reaction-diffusion equations
• Thresholding
• Nomura et al., Patt. Recog. Image Anal., 2008
• Edge detection utilizing reaction-diffusion
  equations
• Adaptive threshold level

18
Integration of Edge Information into
Reaction-Diffusion Stereo Algorithm
ue(x,y), ve(x,y) spatial distributions of
u(x,y,t) and v(x,y,t) at a convergence state
The diffusion coefficients (1-ue) and (1-ve)
suppress propagation of region of the disparity
level dn.
19
Experimental Results

• The Middlebury stereo vision page provides
• stereo image pairs,
• ground-truth data of disparity maps,
• definition of areas (occlusion area depth
  discontinuity),
• scores of other stereo algorithms
• Example of stereo image pairs

TSUKUBA 384X288 pixels 15 disparity levels
CONES 450X375 pixels 60 disparity levels
TEDDY 450X375 pixels 60 disparity levels
VENUS 434X383 pixels 30 disparity levels
20
Edge Detection Result withReaction-Diffusion
Algorithm
ue
Edges detected by a reaction-diffusion algorithm
ve
21
Example CONES
RD (Previous Reaction-Diffusion Algoriothm)
RDIE (Proposed Reaction-Diffusion Algorithm with
Intensity Edge)
22
Experimental Results on the Middlebury Stereo Data
Tested algorithms
Table of scores
RD Nomura et al., Mach. Vis. Appl. (in
press) RDIE Proposed in this presentation. ABP
Klaus et al., Proc. 18th ICPR, 2006. Adaptive
back propagation algorithm Color segmentation
Image Area RD RDIE ABP
CONES nonocc. 5.53 5.36 2.48
CONES all 12.57 12.98 7.92
CONES disc. 15.18 14.45 7.32
TEDDY nonocc. 14.85 14.68 4.22
TEDDY all 20.86 20.91 7.06
TEDDY disc. 30.15 29.06 11.8
TSUKUBA nonocc. 5.98 8.46 1.11
TSUKUBA all 7.86 10.19 1.37
TSUKUBA disc. 19.70 20.13 5.79
VENUS nonocc. 2.75 2.93 0.10
VENUS all 3.92 4.09 0.21
VENUS disc. 21.23 20.18 1.44
Error measure Bad-Match-Percentage () threshold
level 1.0 (pixel)
23
Disparity Error Maps and Intensity Edges
RD
RDIE
Definition of areas
Intensity edges
24
Conclusion Future Work

• Conclusion
• We proposed integration of intensity edge
  information into the reaction-diffusion stereo
  algorithm.
• We confirmed performance of the proposed
  algorithm.
• Future work
• dynamic integration
• other visual cues

Acknowledgments The present study was supported
in part by the Grant-in-Aid for Scientific
Research (C) (No. 20500206) from the Japan
Society for the Promotion of Science.