Einfhrung in die erweiterte Realitt Bildverarbeitung:Tracking Prof. G. Klinker, Prof. B. Brgge 7. Ju

1
Einführung in die erweiterte Realität
BildverarbeitungTracking Prof. G. Klinker,
Prof. B. Brügge7. Juli 2000
2
General description

• Initialization phase
• Determine/place trackable features in the scene
• Generate feature descriptions (3D, 2D, )
• Get initial measurements
• Tracking phase using suitable detectors, past
  user behavior (tracks), and motion models
• predict feature locations at next time interval
• redect features
• update motion models according to new measurements

3
Different Tracking Modalities

• Magnetic
• Mechanical
• Inertial
• GPS
• Ultrasound
• Optical (stationary or mobile camera)
• Hybrid

4
Important Criteria

• Speed
• Precision
• Robustness
• Installation cost(financial, time)
• Usability
• Generality

5
Comparison of Different Tracking Modalities
S P R G W Restrictions
Optical
- -- line of sight
Magnetic
- -- - no metals
Acoustic
- - multiple echoes
Inertial
- drift
CompasTilt
- -- no metals, earth
magnetism
dGPS
-- -- visible to at
least 4 satellites
Mechanic
-- - restricted motions
G Geometric range that is covered W Wireless?
S Speed, lag P Precision, repeatability R
Robustness under rapid motions
6
Optical TrackingWhy not just Calibration?

• Cons (against tracking)
• Extra system complexity
• Motion hysteresis

• Pros (for tracking)
• Reduced search space
• Smooth user motion
• Rendering into the future

7
Tracking loop

• 3D Motion model
• 2D Motion model(s)
• Measurements offeature f in previous i images

m (ft-i ft-1)
Estimate
ft-i ft-1
8
Tracking loop

• 3D Motion model
• 2D Motion model(s)
• Measurements offeature f in previous i images

m (ft-i ft-1)
Predict
Estimate
ft-i ft-1
ft
9
Tracking loop

• 3D Motion model
• 2D Motion model(s)
• Measurements offeature f in previous i images

m (ft-i ft-1)
Predict
Estimate
ft-i ft-1 ft
ft
Search
10
Easily Trackable Targets
2n IDs
cr IDs
11
2D Feature Tracking
12
Target Detection and Identification

• Find dark blobs onbright background
• Fit quadrilateralpolygons
• Find corners
• Read ID label

13
2D Motion Estimation

• Compute local motionvectors of everyfeature
  (images n-1,n-2)
• (more images toestimate higher-order motion
  models)

14
2D Motion Prediction

• Prediction of localfeature motion forimage n

15
Target Redetection

• Prediction of localfeature motion forimage n

16
Recalibration

• Here no explicit 3D motion model!
• Fast
• Reacts quickly to changing user motions (eg head
  shaking)
• Jittery augmentations due to noisy image
  calibration
• Algorithmic options
• Complete recalibration
• Compute motion parameters disjunctly (e.g. first
  rotation parameters, then translation parameters)
• Use previous motion parameters as initial guesses
  for new estimation (hill-climbing Tsai).

17
Other 2D Tracking Techniques
18
Feature Correlation

• unnormalized r(i,j) S t(x,y)
  s(ix,jy)
• normalized

St(x,y)-Ts(ix,jy)-Sij
r(i,j)
S t(x,y) - T2 S s(ix,jy) - Sij2
Image 2
Image 1
j
i
template t, mean T
search region s mean Sij (i,j)
19
Gradient-Based Image Flow
Velocity field in the image plane due to the
motion of the observer, objects or apparent
motion
I(x1,t) - I(x1,t1) image gradient
I(x,t)
u dx
t
t1
v dy ...
I(x1,t)
I(x1,t1)
x1
x2
x
dx
20
Gradient-Based Image Flow

• Aperture problem
• Two variables (u,v) per pixel only one constraint

Ix,y,t
flowu,v flowdx/dt,dy/dt
21
Gradient-Based Optical Flow

• Smoothness assumptionThe velocity field varies
  smoothly over an image.

P D
u uaverage - Ix v vaverage - Iy P Ix
uaverage Iy vaverage It D l2 Ix2 Iy2
P D
22
Gradient-Based Image Flow

• Variational calculus
• Image-flow constraint flow(x,y,t) Exu Eyv
  Et 0
• Smoothness constraint
• Minimize

2
2
2
2
du du dv dv
sm(x,y,t)
dx dy



dx dy dx dv
2
2
(flow(x,y,t) l smoothness(x,y,t)) dx dy
23
3D Tracking Techniques
24
Kalman Filtering

• Physical model of 3D user (camera) motion

25
Kalman Filtering

• Physical motion model Koller circular motion

tx v cos f ty v sin f f w v 0 w 0
26
Kalman Filtering

• Linear filter
• Least squares weighted fit to estimate best
  model parameters from sensor measurements
• Recursive (linear regression)

27
Kalman Filtering
y(n)
C(n)
S
z-1t
v1(n)
x(n)
S
v2(n)
F(n1,n)
v1(n) process noise x(n) state
vector F(n1,n) state transition matrix
v2(n) measurement noise C(n) measurement
matrix y(n) observed data
28
Epipolar Geometry
29
Echtzeit Stereosystem
30
Verfolgung bewegter Objekte
Reale Welt
Sensordaten
Aktionen
Computer
Kommandos
Benutzer
Augmentations
31
Bewegte Objekte mit Marken
32
Model-basierte dynamische Szenenanalyse
33
(No Transcript)
34
n
-
k1
a bc d ef g hij k l m n o p qr s t u v w x y
z ABCDEFGHIJK LM NO PQR S TU V W XYZ