Foundations of ATR with 3-D Data (1)

1
A Mathematical Theoryof Automatic Target
Recognition
Aaron D. Lanterman
(lanterma_at_ece.gatech.edu)
2
What Makes ATR Harder than Factoring Large
Numbers?

• Factoring large numbers may be NP-hard, but...
• At least its easy to precisely specify what the
  problem is!
• Not so easy in ATR
• Subject to controversy

3
Can You Build an Airplane Without a Theory of
Aerodynamics?

• Sure! Without aerodynamic theory, you can do
  this...

• but with a theory, you can do this!

4
Can You Build an Communication Systems w/out
Information Theory?

• Sure! Without Information Theory, you can do this

• but with Information Theory, you can do this!

5
Steam Engines and Thermodynamics

• Dick Blahut likens the situation to steam engines
  coming before the science of thermodynamics
• First steam engines build by entrepreneurs and
  inventors
• Thomas Savery 17th and 18th centuries
• Necessity the mother of invention!
• Thermodynamics didnt begin to crystallize until
  mid 19th century but with it, you eventually get

6
Shannons Lightning Bolt

• 1948 Claude Shannons A Mathematical Theory of
  Communication (1948)
• Later renamed The Mathematical Theory of
  Communication
• Found fundamental limits on what is possible,
  i.e. channel capacity

• Before Shannon, your boss might ask you to do the
  impossible, and fire you if you failed to do it!
• Your boss cannot fire your for failing to exceed
  channel capacity!
• You can tell your boss you need a better channel

7
Theory and Technology

• Advances in theory are not enough
• also need the technology
• Aerodynamic theory alone wont get you a B-2
• need advances in materials, manufacturing
• Information theory along wont get you cell
  phones
• need fast DSP chips, good batteries, even
  more theory (i.e. coding theory)

• Theory tells you whats possible, but sometimes
  only hints at how to get there
• Quantum computing folks does this sound familiar?

8
Info-Theoretic View of ATR
(Statistical Estimation-Theoretic)

Target Recognizer
Scene Understanding
Source
Channel
Decoder
Performance
Bounds
Optimality Criteria
Miss, false alarm rate Confusion
matrices Bias, Variance, M.S.E.
Hypothesis testing (LRT, GLRT) ML, Bayes,
Neyman Pearson Estimation ML, MAP,
M.M.S.E., Bayes
Chernoff Steins Lemma Cramer-Rao
CIS/MIM
9
What Makes ATR Harder than Designing a Cell
Phone?

• The space of X for real-world scenes is extremely
  complicated
• You dont get to pick p(x)
• Likelihood p(yx) is difficult to formulate
• The channel is often deliberately hostile
• Targets hiding in clutter
• Using decoys and camouflage
• Radars can be subject to jamming

10
Variability in Complex Scenes

• Geometric variability
• Position
• Orientation
• Articulation
• Fingerprint
• Environmental variability
• Thermal variability in infrared
• Illumination variability in visual
• Complexity variability
• Number of objects not known

11
Ulf Grenander

• Student of Cramér (yes, that Cramér)
• PhD on statistical inference in function spaces
  (1950)
• Toeplitz Forms and their Applications (with
  Szegö)
• Fundamental work on spectral estimation (1958)
• Probabilities on Algebraic Structures (1968)
• Tutorial on Pattern Theory - unpublished
  manuscript
• Inspired classic paper by Geman Geman (1983)

12
General Pattern Theory

• Generalize standard probability, statistics, and
  shape theory
• Put probability measures on complex structures
• Biological structures
• Mitochondria
• Amoebas
• Brains
• Hippocampus
• Natural language
• Real-world scenes of interest in ATR

13
The 90s GPT Renaissance

• Made possible by increases in computer power
• Michael Miller (Washington Univ., now at JHU) did
  a sabbatical with Grenander
• Fields Medalist David Mumford moves from Harvard
  to Brown shifts from algebraic geometry to
  pattern theory

14
Composite Parameter Spaces

• Naturally handles obscuration
• Dont know how many targets are in the scene in
  advance

• Move away from thinking of detection, location,
  recognition, etc. as separate problems

15
Applying the Grenander Program (1)

• Take a Bayesian approach
• Many ATR algorithms seek features that are
  invariant to pose (position and orientation)
• Grenanders Pattern Theory treats pose as
  nuisance variable in the ATR problem, and deals
  with it head on
• Co-estimate pose, or integrate it out
• At a given viewing angle, Target A at one
  orientation may look much like Target B at a
  different orientation
• the nuisance parameter of orientation
  estimation plays a fundamental role in
  determining the bound on recognition -
  Grenander, Miller, Srivastava

U. Grenander, M.I. Miller, and A. Srivastava,
Hilbert-Schmidt Lower Bounds for Estimators on
Matrix Lie Groups for ATR, IEEE Trans. PAMI,
Vol. 20, No. 2, Aug. 1998, pp. 790-802.
16
Applying the Grenander Program (2)

• Develop statistical likelihood
• Data fusion is natural
• At first, use as much of the data as possible
• Be wary of preprocessing edge extraction,
  segmentation etc.
• Processing can never add information
• Data processing inequality from information
  theory

• If you need to extract features, i.e. for
  real-time computational tractability, try to
  avoid as much loss of information as possible

17
Analytic Performance Bounds

• Estimation bounds on continuous parameters
• Cramér-Rao bounds for continuous pose parameters
• Hilbert-Schmidt metrics for orientation
  parameters
• Bounds on detection/recognition probabilities
• Steins Lemma, Chernoff bounds
• Asymptotic analysis to approximate probabilities
  of error
• Performance in a binary test is dominated by a
  term exponential in a distance measure between a
  true and an alternate target
• Adjust pose of alternate target to get closest
  match to true target as seen by the sensor
  system
• Secondary term involving CRB on nuisance
  parameters
• Links pose estimation and recognition performance

• Anuj
• Srivastava

U. Grenander, A. Srivastava, and M.I. Miller,
Asymptotic Performance Analysis of Bayesian
Target Recognition, IEEE Trans. Info. Theory,
Vol. 46, No. 4, July 2000, pp. 1658-1665.
18
Reading One of DARPAs BAAs

• DARPAs E3D program seeks
• efficient techniques for rapidly exploiting 3-D
  sensor data to precisely locate and recognize
  targets.
• BAA full of demands (hopes?) for different stages
  of the program, such as
• The Target Acquisition and Recognition
  technology areas will develop techniques to
  locate and recognize articulating, reconfigurable
  targets under partial obscuration conditions,
  with an identification probability of 0.85, a
  target rejection rate less than 5, and a
  processing time of 3 minutes per target or less

19
Leads Us to Wondering

• If such a milestone is not reached,
• is that the fault of the algorithm or the
  sensor?
• How does the DARPA Program Manager know who to
  fire?
• Without a theory, the DARPA PM may fire someone
  who was asked to exceed channel capacity, i.e.
  given an impossible task
• What performance from a particular sensor is
  necessary to achieve a certain level of ATR
  performance,
• independent of the question of what
  algorithm is used?

20
Perspective Projection
21
Sensor Effects
22
Loglikelihood

• CCD loglikelihood of Snyder et. al

where

• Cascade with

• Sensor fusion natural just add loglikelihoods

23
Langevin Diffusion Processes

• Write posterior in Gibbs form

• Fix number of targets and target types
• Simulate Langevin diffusion

• Distribution of

• Computed desired statistics from the samples
• Generalizes to non-Euclidean groups like
  rotations
• Gradient computation
• Numeric approximations
• Easy and fast on modern 3-D graphics hardware

24
Jump Processes
Death
Type-change
Birth
25
Jump Strategies

• Gibbs style
• Sample from a restricted part of the posterior
• Metropolis-Hastings style
• Draw a proposal from a proposal density
• Accept (or reject) the proposal with a certain
  probability

26
Example Jump-Diffusion Process
27
Thermal Variability
Simulations from PRISM Discretizes target
surface using regions from CAD template and
internal heat transfer model
Average Static State
Average Dynamic State
CIS/MIM
28
Cant Hide from Thermal Variations
Profile 8 Profile 45 Profile
75 Profile 140
Performance Variations Due To Thermodynamic
Variability
Performance Loss Due To Inaccurate Thermodynamic
Information
Cooper, Miller SPIE 97
CIS/MIM
29
Principle Component Representation of Thermal
State

• Model radiance as scalar random field on surface
• Compute empirical mean covariance from database
  of 2000 radiance profiles
• Karhunen-Loeve expansion using eigenfunctions of
  covariance on surface - Eigentanks
• Add expansion coefficients to parameter space
• Fortunately, able to estimate directly given pose

• A younger, much
• thinner Aaron
• Lanterman

• Matt Cooper
• (now with Xerox)

SPIE 97 Cooper, Grenander, Miller, Srivastava
CIS/MIM
30
The First Eigentanks
Meteorological Variation
Operational Variation
Remember, were showing 2-D views of full 3-D
surfaces
Composite Mode of Variation
SPIE 97 Cooper, Grenander,
Miller, Srivastava
CIS/MIM
31
Joint MAP Est. of Pose and Thermal Signature
Real NVESD M60 data (courtesy James Ratches)
Initial Estimate
Final Estimate
CIS/MIM
SPIE 98 Cooper and Miller
32
Cost of Estimating Thermal State
MSE Performance Loss Comanche SNR
5.08 dB
CIS/MIM
33
Ladar/IR Sensor Fusion
MSE Performance Bound
Information Bound
Tom Green
Joe Kostakis
Jeff Shapiro
FLIR (intensity)
LADAR (range)
CIS/MIM
34
LADAR IR Sensor Fusion

LADAR/FLIR Hannon Curve 15 degrees
error
LADAR/FLIR Hannon Curve 9 degrees
error
SPIE 98 Advanced Techniques ATR III
Kostakis, Cooper, Green, Miller,
OSullivan, Shapiro Snyder
CIS/MIM
35
Target Models
Panzer IILight Tank
Sturmgeschultz IIISelf-Propelled Gun
Semovente M41 Self-Propelled Gun
M48 A3 Main Battle Tank
Hull Length 4.81 mWidth 2.28 mHeight 2.15 m
Hull Length 6.77 mWidth 2.95 mHeight 2.16 m
Hull Length 5.205 mWidth 2.2 mHeight 2.15 m
Hull Length 6.419 mWidth 3.63 mHeight 3.086
m
(Info and Top Row of Images from 3-D Ladar
Challenge Problem Slides by Jacobs Sverdrup)
36
CR-Bound on Orientation
Position assumed known
We take a performance hit!
Strum
Position unknown, must be co-estimated
Semo
Interesting knee at 0.2 meters
37
M48 vs. Others
M48 and Panzer have dissimilar signatures most
easily distinguished
M48 and Semo have similar signatures most easily
confused
38
Semovente vs. Others
At higher resolutions, Semo and M48 have most
dissimilar signatures most easily
distinguished (perhaps there are nice features
which only become apparent at higher resolutions?)
At lower resolutions, Semo and Panzer have most
dissimilar signatures most easily distinguished
Semo and Sturm have similar signatures most
easily confused
39
Synthetic Aperture Radar
Michael DeVore
Joseph OSullivan

• MSTAR Data Set
• Conditionally Gaussian model for pixel values
  with variances trained from data
• Likelihood based classification
• Target orientation unknown and uniformly
  distributed over 360 of azimuth
• Joint orientation estimation and target
  classification
• Train on 17 depression angle
• Test on 15 depression angle

T72
BMP 2
Variance Images
SAR Images
CIS/MIM
40
Results using 72 variance images per target of
10 each, and using 80 x 80 pixel sub-images to
reduce background clutter Probability of
correct classification 98 Average
orientation error lt 10
Orientation MSE effects ID!
CIS/MIM
Supported by ARO Center for Imaging Science DAAH
04-95-1-04-94 and ONR MURI N00014-98-1-06-06
41
Caveat
Do not confuse the model with reality.
42
Where Should Clutter Go? (1)
A forward model, i.e. a scene simulator
non-Gaussian minimax entropy texture models by
Song Chun Zhu

• A forest might go well in the noise part

43
Where Should Clutter Go? (2)

• but downtown Baghdad will not whiten
• Structured clutter is the most vexing
• May need to go in here, and directly manipulate
  the clutter

• or a bit of each

• Where to draw the line?

44
Acknowledgments

• Much of the work described here was funded by the
  ARO Center for Imaging Science
• Also ONR (William Miceli) and AFOSR (Jon Sjogren)
• Slides with CIS/MIM tag were adapted from slides
  provided by Michael Miller