Random SetPoint Process in MultiTarget Tracking

1
Random Set/Point Process in Multi-Target Tracking
Ba-Ngu VoEEE Department University of
MelbourneAustralia
http//www.ee.unimelb.edu.au/staff/bv/
Collaborators (in no particular order) Mahler
R., Singh. S., Doucet A., Ma. W.K., Panta K.,
Clark D., Vo B.T., Cantoni A., Pasha A., Tuan
H.D., Baddeley A., Zuyev S., Schumacher D.
SAMSI, RTP, NC, USA, 8 September 2008
2
Outline

• The Bayes (single-target) filter
• Multi-target tracking
• System representation
• Random finite set Bayesian Multi-target
  filtering
• Tractable multi-target filters
• Probability Hypothesis Density (PHD) filter
• Cardinalized PHD filter
• Multi-Bernoulli filter
• Conclusions

3
The Bayes (single-target) Filter
observation space
zk
zk-1
state space
target motion
xk
xk-1
state-vector
System Model
fkk-1(xk xk-1)
gk(zk xk)
Markov Transition Density
Measurement Likelihood
Objective
pk(xk z1k)
measurement history (z1,, zk)
posterior (filtering) pdf of the state
4
The Bayes (single-target) Filter
observation space
zk
zk-1
state space
target motion
xk
xk-1
state-vector
? pk-1(xk-1 z1k-1)
dxk-1
fkk-1(xk xk-1)
gk(zk xk)
K-1 pkk-1(xk z1k-1)
Bayes filter
pkk-1(xk z1k-1)
prediction
data-update
pk(xk z1k)
pk-1(xk-1 z1k-1)
???
???
5
The Bayes (single-target) Filter
observation space
zk
zk-1
gk(zk xk)
state space
target motion
xk
fkk-1(xk xk-1)
xk-1
state-vector
6
Multi-target tracking
7
Multi-target tracking
observation space
observation produced by targets
state space
target motion
Xk
Xk-1
3 targets
5 targets

• Objective Jointly estimate the number and states
  of targets
• Challenges
• Random number of targets and measurements
• Detection uncertainty, clutter, association
  uncertainty

8
System Representation
How can we mathematically represent the
multi-target state?

• Usual practice stack individual states into a
  large vector!
• Problem

True Multi-target state
Estimated Multi-target state
2 targets
2 targets

• Estimate is correct but estimation error
  ???

• Remedy use

9
System Representation
True Multi-target state
Estimated Multi-target State
2 targets
1 target
True Multi-target state
Estimated Multi-target State
2 targets
no target

• What are the estimation errors?

10
System Representation

• Error between estimate and true state
  (miss-distance)
• fundamental in estimation/filtering control
• well-understood for single target Euclidean
  distance, MSE, etc
• in the multi-target case depends on state
  representation

• For multi-target state
• vector representation doesnt admit multi-target
  miss-distance
• finite set representation admits multi-target
  miss-distance distance between 2 finite sets

• In fact the distance
• is a distance for sets not vectors

11
System Representation
observation space
observation produced by targets
state space
target motion
Xk
Xk-1
3 targets
5 targets

• Number of measurements and their values are
  (random) variables
• Ordering of measurements not relevant!
• Multi-target measurement is represented by a
  finite set

12
RFS Bayesian Multi-target Filtering

• Reconceptualize as a generalized single-target
  problem Mahler 94

observed set
observations
??
?Z
target set
targets
?X
?X

• Bayesian Model state observation as Random
  Finite Sets Mahler 94

prediction
data-update
pk(XkZ1k)
pkk-1(XkZ1k-1)
???
pk-1(Xk-1Z1k-1)
???

• Need suitable notions of density integration

13
RFS Bayesian Multi-target Filtering
state space E
S
random finite set or random point pattern
S
state space E
NS(S) S ? S
point process or random counting measure
14
RFS Bayesian Multi-target Filtering
F(E)
E
State space
Collection of finite subsets of E
S
S
T
T
Belief distribution of S bS (T ) P(S Í T ) ,
T Í E
Probability distribution of S PS (T ) P(S ÎT )
, T Í F(E)
Choquet (1968)
Point Process Theory (1950-1960s)
Mahlers Finite Set Statistics (1994)
Belief density of S fS F(E) 0,)
bS (T ) òT fS (X)dX
Probability density of S pS F(E) 0,)
PS (T ) òT pS (X)m(dX)
Vo et. al. (2005)
Conventional integral
Set integral
15
Multi-target Motion Model
Xk Skk-1(Xk-1)ÈBkk-1(Xk-1)ÈGk
fkk-1(Xk?Xk-1 )
Multi-object transition density
Evolution of each element x of a given
multi-object state Xk-1
16
Multi-target Observation Model
Zk Qk(Xk) ÈKk(Xk)
likelihood
x
z
gk(ZkXk)
misdetection
?
Multi-object likelihood
x
clutter
?
state space
observation space
Observation process for each element x of a
given multi-object state Xk
17
Multi-target Bayes Filter
Multi-target Bayes filter
pk(XkZ1k)
pkk-1(XkZ1k-1)
prediction
pk-1(Xk-1Z1k-1)
data-update
???
???

• Computationally intractable in general
• No closed form solution
• Particle or SMC implementation
• Vo, Singh Doucet 03, 05, Sidenbladh 03,
  Vihola 05, Ma et al. 06
• Restricted to a very small number of targets

18
Particle Multi-target Bayes Filter

• Algorithm
• for i 1N, Initialise gt
• Sample
• Compute
• end
• normalise weights
• for k 1 kmax ,
• for i 1N, Update gt
• Sample
• Update
• end
• normalise weights
• resample
• MCMC step
• end

19
The PHD Filter
pk(XkZ1k)
pkk-1(XkZ1k-1)
prediction
pk-1(Xk-1Z1k-1)
data-update
???
???

• Multi-target Bayes filter very expensive!

single-object Bayes filter

state of system random vector
first-moment filter (e.g. a-b-g filter)

• Single-object

multi-object Bayes filter

state of system random set
first-moment filter (PHD filter)

• Multi-object

20
The Probability Hypothesis Density
vS PHD (intensity function) of a RFS S
vS(x)dx expected number
of objects in S
?S
mean of, NS(S), the random counting
measure at S
x0
state space
S
21
The PHD Filter
state space
vk
vk-1

• Avoids data association!

PHD filter
PHD prediction
PHD update
vkk-1(xkZ1k-1)
vk-1(xk-1Z1k-1)
vk(xkZ1k)
???
???
prediction
pk-1(Xk-1Z1k-1)
pk(XkZ1k)
pkk-1(XkZ1k-1)
update
???
???
Multi-object Bayes filter
22
PHD Prediction
?
vkk-1(xk Z1k-1) fkk-1(xk, xk-1)
vk-1(xk-1Z1k-1)dxk-1 gk(xk)
fkk-1(xk, xk-1) ekk-1(xk-1) fkk-1(xkxk-1)
bkk-1(xkxk-1)
probability of object survival
term for objects spawned by existing objects
intensity of Bk(xk-1)
Markov transition density
23
PHD Update
S
1 - pD,k(xk)vkk-1(xkZ1k-1)
pD,k(xk)gk(zxk)
vk(xkZ1k) ?
Dk(z) kk(z)
z?Zk
Bayes-updated intensity
predicted intensity (from previous time)
intensity of false alarms
probability of detection
measurement
?
Dk(z) pD,k(x)gk(zx)vkk-1(xZ1k-1)dx
Nk vk(xZ1k)dx
?
sensor likelihood function
expected number of objects
24
Particle PHD filter

• The PHD (or intensity function) vk is not a
  probability density
• The PHD propagation equation is not a standard
  Bayesian recursion
• Sequential MC implementation of the PHD filter

Vo, Singh Doucet 03, 05, Sidenbladh 03,
Mahler Zajic 03
state space
Particle approximation of vk
Particle approximation of vk-1

• Need to cluster the particles to obtain
  multi-target estimates

25
Particle PHD filter

• Algorithm
• Initialise
• for k 1 kmax ,
• for i 1 Jk ,
• Sample compute
  
• end
• for i Jk 1 Jk Lk-1 ,
• Sample compute
  
• end
• for i 1 Jk Lk-1 ,
• Update
• end
• Redistribute total mass among Lk resampled
  particles
• end

Convergence Vo, Singh Doucet 05, Clark
Bell 06, Johansen et. al. 06
26
Gaussian Mixture PHD filter

• Closed-form solution to the PHD recursion exists
  for linear Gaussian multi-target model

• Gaussian mixture prior intensity Þ Gaussian
  mixture posterior intensities at all subsequent
  times

PHD filter
Gaussian Mixture (GM) PHD filter Vo Ma 05, 06

• Extended Unscented Kalman PHD filter Vo Ma
  06
• Jump Markov PHD filter Pasha et. al. 06
• Track continuity Clark et. al. 06

27
Cardinalised PHD Filter

• Drawback of PHD filter High variance of
  cardinality estimate

• Relax Poisson assumption allows arbitrary
  cardinality distribution

• Jointly propagate intensity function
• probability generating
  function of cardinality.

intensity prediction
intensity update
vkk-1(xkZ1k-1)
vk-1(xk-1Z1k-1)
vk(xkZ1k)
???
???
CPHD filter Mahler 06,07

• More complex PHD update step (higher
  computational costs)

28
Gaussian Mixture CPHD Filter

• Closed-form solution to the CPHD recursion exists
  for linear Gaussian multi-target model

• Gaussian mixture prior intensity Þ Gaussian
  mixture posterior intensities at all subsequent
  times Vo et. al. 06, 07

• Particle-PHD filter can be extended to the CPHD
  filter

cardinality prediction
cardinality update
??
??
??
pkk-1(n)
pk(n)
pk-1(n)
???
???
n0
n0
n0
intensity prediction
intensity update
Jk
Jk-1
Jkk-1
(i)
(i)
(i)
(i)
(i)
(i)
wk-1, xk-1
wkk-1, xkk-1
wk, xk
???
???
i1
i1
i1
Particle CPHD filter Vo 08
29
CPHD filter Demonstration
1000 MC trial average
GMCPHD filter
GMPHD filter
30
CPHD filter Demonstration
Comparison with JPDA linear dynamics, sv 5,
sh 10, 4 targets,
1000 MC trial average
31
CPHD filter Demonstration
Sonar images
32
MeMBer Filter
Multi-object Bayes filter
prediction
pk-1(Xk-1Z1k-1)
pk(XkZ1k)
pkk-1(XkZ1k-1)
update
???
???
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
(Multi-target Multi-Bernoulli ) MeMBer filter
Mahler 07, biased
Cardinality-Balanced MeMBer filter Vo et. al.
07, unbiased

• Approximate predicted/posterior RFSs by
  Multi-Bernoulli RFSs

• Valid for low clutter rate high probability of
  detection

33
Cardinality-Balanced MeMBer Filter
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
Mk-1
MG,k
(i)
(i)
(i)
(i)
(rP,kk-1, pP,kk-1) È (rG,k, pG,k)
i1
i1
term for object births
(i)
á fkk-1(), pk-1 pS,kñ
Cardinality-Balanced MeMBer filter Vo et. al.
07
(i)
ápk-1, pS,kñ
34
Cardinality-Balanced MeMBer Filter
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
Mkk-1
(i)
(i)
(rL,k, pL,k) È (rU,k,(z), pU,k(z))
zÎZk
i1
pD,kgk(z)
Mkk-1
rkk-1(1- rkk-1) ápkk-1, pD,kgk(z)ñ
(i)
(i)
(i)
S
(i)
(1- rkk-1ápkk-1, pD,kñ)2
(i)
i1
Cardinality-Balanced MeMBer filter Vo et. al.
07
k(z)
35
Cardinality-Balanced MeMBer Filter
Closed-form (Gaussian mixture) solution Vo et.
al. 07,
Jk-1
Jkk-1
Jk
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
wkk-1, mkk-1, Pkk-1
wk-1, mk-1, Pk-1
wk, mk, Pk
j1
j1
j1
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
Jk-1
Jkk-1
Jk
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
(i,j)
wkk-1, xkk-1
wk-1, xk-1
wk, xk
j1
j1
j1
Particle implementation Vo et. al. 07,
More useful than PHD filters in highly non-linear
problems
36
Performance comparison
Example 10 targets max on scene, with
births/deaths 4D states x-y
position/velocity, linear Gaussian observations
x-y position, linear Gaussian
Dynamics constant velocity model ?v
5ms-2, survival probability pS,k
0.99, Observations additive Gaussian noise
?? 10m, detection probability pD,k
0.98, uniform Poisson clutter ?c
2.5x10-6m-2
?/? start/end positions
37
Gaussian implementation
1000 MC trial average
Cardinality-Balanced Recursion
Mahlers MeMBer Recursion
38
Gaussian implementation
1000 MC trial average
CPHD Filter has better performance
39
Particle implementation
1000 MC trial average
CB-MeMBer Filter has better performance
40
Concluding Remarks

• Random Finite Set framework
• Rigorous formulation of Bayesian multi-target
  filtering
• Leads to efficient algorithms
• Future research directions
• Track before detect
• Performance measure for multi-object systems
• Numerical techniques for estimation of
  trajectories

For more info please see http//randomsets.ee.unim
elb.edu.au/
Thank You!
41
References

• D. Stoyan, D. Kendall, J. Mecke, Stochastic
  Geometry and its Applications, John Wiley Sons,
  1995
• D. Daley and D. Vere-Jones, An Introduction to
  the Theory of Point Processes, Springer-Verlag,
  1988.
• I. Goodman, R. Mahler, and H. Nguyen, Mathematics
  of Data Fusion. Kluwer Academic Publishers, 1997.
• R. Mahler, An introduction to multisource-multita
  rget statistics and applications, Lockheed
  Martin Technical Monograph, 2000.
• R. Mahler, Multi-target Bayes filtering via
  first-order multi-target moments, IEEE Trans.
  AES, vol. 39, no. 4, pp. 11521178, 2003.
• B. Vo, S. Singh, and A. Doucet, Sequential Monte
  Carlo methods for multi-target filtering with
  random finite sets, IEEE Trans. AES, vol. 41,
  no. 4, pp. 12241245, 2005,.
• B. Vo, and W. K. Ma, The Gaussian mixture PHD
  filter, IEEE Trans. Signal Processing, IEEE
  Trans. Signal Processing, Vol. 54, No. 11, pp.
  4091-4104, 2006.
• R. Mahler, A theory of PHD filter of higher
  order in target number, in I. Kadar (ed.),
  Signal Processing, Sensor Fusion, and Target
  Recognition XV, SPIE Defense Security
  Symposium, Orlando, April 17-22, 2006
• B. T. Vo, B. Vo, and A. Cantoni, "Analytic
  implementations of the Cardinalized Probability
  Hypothesis Density Filter," IEEE Trans. SP, Vol.
  55,  No. 7,  Part 2,  pp. 3553-3567, 2007.
• D. Clark J. Bell, Convergence of the
  Particle-PHD filter, IEEE Trans. SP, 2006.
• A. Johansen, S. Singh, A. Doucet, and B. Vo,
  "Convergence of the SMC implementation of the PHD
  filter," Methodology and Computing in Applied
  Probability, 2006.
• A. Pasha, B. Vo, H. D Tuan and W. K. Ma,
  "Closed-form solution to the PHD recursion for
  jump Markov linear models," FUSION, 2006.
• D. Clark, K. Panta, and B. Vo, "Tracking multiple
  targets with the GMPHD filter," FUSION, 2006.
• B. T. Vo, B. Vo, and A. Cantoni, On
  Multi-Bernoulli Approximation of the Multi-target
  Bayes Filter," ICIF, Xian, 2007.
• See also http//www.ee.unimelb.edu.au/staff/bv/p
  ublications.html

42
Representation of Multi-target state
Optimal Subpattern Assignment (OSPA) metric
Schumacher et. al 08

• Fill up X with n - m dummy points located at a
  distance greater than c from any points in Y
• Calculate pth order Wasserstein distance between
  resulting sets
• Efficiently computed using the Hungarian algorithm

43
Gaussian Mixture PHD Prediction

• Gaussian mixture posterior intensity at time
  k-1

• Gaussian mixture predicted intensity to time k

44
Gaussian Mixture PHD Update

• Gaussian mixture predicted intensity to time k

• Gaussian mixture updated intensity at time k

45
Cardinalised PHD Prediction
pkk-1(n) p??,k(n - j)
??kk-1vk-1,pk-1(j)
predicted cardinality
probability of n - j spontaneous births
probability of j surviving targets
Cjl ltpS,k ,vk-1gt j lt1- pS,k ,vk-1gt l-j
pk-1 (l)
lt1,vk-1gtl
?
vkk-1(xk) pS,k(xk-1) fkk-1(xkxk-1)
vk-1(xk-1)dxk-1 gk(xk)
intensity of spontaneous object births Gk
probability of survival
Markov transition density
predicted intensity
intensity from previous time-step
46
Cardinalised PHD Update
0
?kvkk-1, Zk(n)pkk-1(n)
pk(n)
0
ltkvkk-1, Zk, pkk-1gt
predicted cardinality distribution
updated cardinality distribution
47
Mahlers MeMBer Filter
Multi-object Bayes filter
prediction
pk-1(Xk-1Z1k-1)
pk(XkZ1k)
pkk-1(XkZ1k-1)
update
???
???
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
(Multi-target Multi-Bernoulli ) MeMBer filter
Mahler 07

• Approximate predicted/posterior RFSs by
  Multi-Bernoulli RFSs

• Valid for low clutter rate high probability of
  detection

• Biased in Cardinality (except when probability of
  detection 1)

48
Cardinality-Balanced MeMBer Filter
prediction
update
Mk-1
Mkk-1
(i)
(i)
Mk
(i)
(i)
(i)
(i)
(rk-1, pk-1)
(rkk-1, pkk-1)
(rk, pk )
???
???
i1
i1
i1
Cardinality-Balanced MeMBer filter Vo et. al.
07
Mkk-1
(i)
(i)
(rL,k, pL,k) È (rU,k,(z), pU,k(z))
zÎZk
i1
49
Extensions of the PHD filter

• Linear Jump Markov PHD filter Pasha et. al. 06

50
Extensions of the PHD filter
Example 4-D, Linear JM target dynamics with 3
models 4 targets, birth rate 3x0.05, death prob.
0.01, clutter rate 40
51
What is a Random Finite Set (RFS)?
Pine saplings in a Finish forest Kelomaki
Penttinen
Childhood leukaemia lymphoma in North
Humberland Cuzich Edwards

• The number of points is random,
• The points have no ordering and are random
• Loosely, an RFS is a finite set-valued random
  variable
• Also known as (simple finite) point process or
  random point pattern