Estimation of local vector field from High resolution image for forestry application

1
Estimation of local vector field from High
resolution image for forestry application
LIAMA Master degrees March-August 2005 Author
Cassisa Cyril
Tutor Prinet Véronique
2
Table of contents

• Introduction
• Scientific objectives
• Existing works and theory
• Bibliography
• Theory
• Optical flow
• Markov
• My research
• Problem statement
• Proposed approaches
• Optimization
• Results
• Perfect synthetic sequence
• Simulated forestry images
• Constant displacement
• Unidirectional displacement
• Radial displacement
• Other application
• Conclusion

3
Introduction

• Trees are moving
• Local movement
• space constraints
• light environment
• Global movement
• a landslide
• a strong wind.
• Main Idea
• Understand the movement of trees to prevent
  the risk of landslide

Example of landslide in China
4
Scientific objectives

• Development of probabilistic models (MRF,DRF)
  for object displacement estimation from remote
  sensing images sequence
• Characteristics to be tracked in the images
• Hypothesis on the displacement

5
Context and application

• Application
• Displacement of the forest trees (!)
• Related projects
• ARC mode de vie (Ariana-Digiplante) - A(Marked
  point process)
• Seed project ChinaDSS (INRA-LIAMA)
  S(Application)
• PRA (Vista-LIAMA )
  - S(Modeling the movement)

6
Existing works and theory

• Bibliography
• Optical flow
• Markov random field
• Theory
• Optical flow
• Markov random field

7
Bibliography

• Optical flow
• B. Horn and B. Schunck, Determining Optical Flow,
  Artificial Intelligence, vol.17, 1981
• J. Weickert, A. Bruhn, C. Schnorr, Combining the
  advantage of local and global optical flow
  methods, Tech. Report, Univ. of Mannheim
  (Germany), April 2004.
• J.L. Barron, D.J. Fleet, S.S. Beauchemin,
  Performance of Optical Flow Techniques, Dept. Of
  Computer Science, Ontario 1994
• Lucas/Kanade meet Horn/Shunck, International
  Journal of Computer Vision, vol 61, 2005.

8
Bibliography

• Markov random field
• F.Heitz and P.Bouthemy, Multi-modal estimation of
  discontinuous optical flow using markov random
  fields, IEEE Trans. Pattern Analysis and Machine
  Intelligence, December 1993, vol.15 no12, p.
  1217-1232
• Stan Z.Li, Markov Random Field Modeling in Image
  Analysis, 2nd ed., Springer-Verlag, 2001.
• Julian Besag, On the Statistical Analysis of
  Dirty Pictures, Journal of the Royal Society, N3
  (1986)
• Etienne Memin, Patrick Perez, Hierarchical
  Estimation and Segmentation of dense Motion
  Fields, July 2001.

9
Theory - Optical flow
Definition Computation of local displacement
between an image t1 and an image t2

• Image sequences real function I(X,t) with
  X(x,y)
• Hypothesis of constant illumination
• Small displacement between 2 images.

Optical flow equation
vector displacement
2 unknown values (u,v) for 1 equation ? need to
add regularization
10
Optical flow - illustration
Optical flow on black square moving with a
diagonal displacement using Horn Shunck model
11
Theory - Markov Random Fields

• Sites, Neighborhoods, Cliques
• Markov definition and equations

12
Markov Random Fields
In Markov, we consider we have a graph which is
composed by sites (nodes) where there is existing
relations between these sites

• Sites s
• Neighborhoods Ns
• Cliques

Nodes of the Images (pixel, objects) Here there
is 9 sites s. Let take an example for the site n5
Depends of the considered connexity Here we use
a 4-neighbourhood connexity
A clique c is defined as a subset of site in
S. The clique has an order.
Example on the site n5
13
Markov Random Fields

• Definition
• Markov-Gibbs equivalence

Family of random variable
Local dependency
Gibbs distribution
The objective is to find the configuration f
that maximizes the probability
?minimizes the energy
14
My research

• Problem statement
• Proposed approaches
• Optimization

15
Problem statement

• Optical flow Markov
• Search the best configuration of vector
  displacement wwi(ui,vi)
• maximizing joint probability P(w, Ix,Iy, It)
• - Data term of the Gibbs energy is given by the
  optical flow equation.
• - Prior knowledge term (constraints) should
  satisfy our hypothesis between neighborhood
  cliques.
• Then,
• the joint probability is given by

16
Proposed approaches

• Dense approach (sppixel)
• Object approach 1 object (sotree)
• Object approach 2 object (sotree) with
  constraint on pixel of an object

17
Dense approach (sppixel)

• Hypothesis
• Vector displacement is computed locally (at each
  pixel) C1sp (i,j) C2sp,sp
• no restriction on the collinearity of the
  displacement vectors at two neighborhood, only a
  smoothness restriction.

18
Object approach 1 object (sotree)

• Hypothesis
• A object-tree defines a site s, with a 1st order
  neighborhood C1so C2so,so
• The displacement at an object is approximated by
  the average displacement computed from all pixels
  that constitute the object.
• Between 2 trees we have
• Smoothness constraint E3
  Parallelism constraint E4

19
Object approach 2

• Hypothesis
• S1 denotes the set of pixel sites in the image
  C1sp (i,j) C2sp,sp
• S2 the set of tree sites in the image C3
  so,so with so the neighborhood tree of so
• - new definition of velocity global local
  (noise) movements
• The displacement at an object is approximated the
  displacement of the pixels inside the object
  respecting the constraint (E2) of the white
  noise.
• - Between 2 trees, we always have Parallelism
  (E4) and Smoothness constraints (E3)

with
If s is inside the object
Otherwise
,
with
Ponderation terms
20
Optimization

• ICM algorithm
• Principe Find local minimum of the energy
• Method
• Initialisation (u,v) (0,0)
• Random generation of u(n), v(n) between -55
• Computation of the energy.
• Tend to a local minimum of energy using iterations

21
Results

• Perfect synthetic sequence
• Simulated forestry images
• Constant displacement
• Unidirectional displacement
• Radial displacement
• Other application

22
Perfect synthetic sequence

• Input data

1st displacement
2nd displacement
23

• Result illustrations 1st displacement

P 1000 iterations
Variance
Angle error (degrees) Ideal, estimated vectors
O1 100000 iterations
Quantitative values on the 3 approaches
O2 100000 and after 1000 iterations

• Object approaches better than pixel approach

24

• Result illustrations 2nd displacement

pixel approach (P) 1000 iterations
object 1 approach (O1) 100000 iterations
Quantitative values on the 3 approaches

• Object approaches more robust than pixel approach

object 2 approach (O2) 100000 and after 1000
iterations
25
Simulated forestry images

• Constant displacement
• Unidirectional displacement
• Radial displacement

Forestry image generated by AMAP
26
Tree detection
Detection representation of trees using the tree
foot coordinates.
27
Constant displacement

• Input data

Image on which we applied a vertical movement of
2 pixel
Representation of the ideal movement
G.Perrin image synthetic forest
28
Constant displacement

• Result illustrations

O1
P
O2
Quantitative values on the 3 approaches
29
ICM limitation
Different estimation of optical flow using the
normal object approach 1 All the results have
the same parameters and the same condition of
iteration.
A
B
? ICM finds local minimums The results depend of
the initialization
C
D
30
Unidirectional displacement

• Input data

Terrain topology
Synthetic displacement of the tree feet
Quantity of displacement of the trees from the
top (right) of the slope to the bottom
31
Unidirectional displacement

• Result illustrations

O2
O1
P
Statistics between the angle of the velocity of
tree foot and the estimation of velocity
32
Estimation of velocity using OpenCV HornShunck
algorithm
Estimation of velocity using Barron HornShunck
algorithm
Estimation of velocity using our object approach 1
Estimation of velocity using our object approach 2
33
radial displacement

• Input data

Model of the radial displacement
Synthetic displacement of the tree feet
34
Estimation of velocity using OpenCV HornShunck
algorithm
Estimation of velocity using Barron HornShunck
algorithm
Estimation of velocity using our object approach 1
Estimation of velocity using our object approach 2
35
Other application

• Rubik cube sequence

LucasKanade (Barron) ( 55)
Rubik cube image
HornShunck (Barron) ( 2, )
object 2 approach (O2)
36
Conclusion

• We propose an object based approach to estimate
  the tree displacement.
• On our images, the object approach seems more
  robust than the others
• We notice 2 main problems
• We need a better optimization
• Use a MCMC algorithm which computes the MAP and
  not local minimum as ICM
• The images do not satisfy the hypothesis of
  constant intensity.
• There is many possible extensions of this work to
  the computation of displacement field from any
  type of video sequences (ex car tracking)

37
(No Transcript)
38
LIAMA
January 1997 Chinese Academy of Sciences and
INRIA signed the cooperation agreements which set
up the French-Chinese Laboratory on Computer
Science, Automation and Applied Mathematics.
LIAMA is a only French-Chinese Laboratory on
image processing in China. Current "On Site
Projects" RSIU (Remote Sensing Data Fusion and
Image Understanding with applications to
environmental problems) GreenLab (Structural
and Functional Modeling, Simulation and
Visualization of Plant Architecture and Growth)
CAD (Computer Graphics and Geometry) Scilab
Open source promotion
Institute of Automation of the Chinese Academy of
Sciences half of the 11th floor is used by
LIAMA
39
AMAP

• AMAP is a software which simulates the plants
  behaviour using various factors.
• In this domain there is 2 generation of
  softwares
• The 1st generation (L-System, Visual Plant,
  Lignum...)
• The 2nd (AMAPsim, AMAPhydro)
• AMAP was developed in CIRAD and was based on an
  automaton system that decomposed a plant in
  substructures.

40
Implementation and algorithms

• Dense
• Object 1

Computation of Ix(sp), Iy(sp), It(sp) for each
pixel (i,j) Equation n1 Dense
approach Retrieve u(sp), v(sp) using
ICM Visualization
Computation of Ix(sp), Iy(sp), It(sp) for each
pixel (i,j) Computation of
for each object s Equation n2 Object
approach 1 Retrieve ug(so), vg(so) using
ICM Visualization
41
Implementation and algorithms

• Object 2

Run first the Object approach 1 Estimate
ug(so), vg(so) for each object s Equation n3
Object approach 2 Retrieve u(sp), v(sp) for each
pixel (i,j) in the object s using
ICM Visualization