Robust Statistics Why do we use the norms we do

1
Robust StatisticsWhy do we use the norms we do?

• Henrik Aanæs
• IMM,DTU
• haa_at_imm.dtu.dk

A good general reference is Robust Statistics
Theory and Methods, by Maronna, Martin and Yohai.
Wiley Series in Probability and Statistics
TexPoint fonts used in EMF. Read the TexPoint
manual before you delete this box. AAAAA
2
How Tall are You ?
3
Idea of Robust Statistics
To fit or describe the bulk of a data set well
without being perturbed (influenced to much) by
a small portion of outliers. This should be done
without a pre-processing segmentation of the data.
Outliers can be interesting too!
4
Line Example
5
Robust Statistics in Computer VisionImage
Smoothing
Image by Frederico D'Almeida
6
Robust Statistics in Computer VisionImage
Smoothing
7
Robust Statistics in Computer Visionoptical flow
Play Sequence MIT BCS Perceptual Science Group.
Demo by John Y. A. Wang.
8
Robust Statistics in Computer Visiontracking via
view geometry
Image 1
Image 2
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
Gaussian/ Normal DistributionThe Distribution We
Usually Use

• Nice Properties
• Central Limit Theorem.
• Induces two norm.
• Leads to linear computations.

• But
• Is fiercely influenced by outliers.
• Empirical distributions often have fatter
  tails.

13
Gaussians Just are Models Too

• Alternative title of this talk

14
Error or ?-functionsConverting from Model-Data
Deviation to Objective Function.
15
?-functions and MLA typical way of forming ?-
functions
16
?-functions and ML IIA typical way of forming ?-
functions
17
?-functions and ML IIA typical way of forming ?-
functions
18
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

General Idea Down weigh outliers, i.e. ?(x)
should be smaller for large x.
19
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Induced by Gaussian.
• Very non-robust.
• Standard distribution.

20
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Quite Robust.
• Convex.
• Corresponds to Median.

21
The Median and the 1-Norm
22
The Median and the 1-NormExample with 2
observations
23
The Median and the 1-NormExample with 2
observations
24
The Median and the 1-NormExample with more
observations
25
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Quite Robust.
• Convex.
• Corresponds to Median.

26
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Mixture of 1 and two norm.
• Convex.
• Has nice theoretical properties.

27
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Discards Outliers.
• For inliers works asGaussian.
• Has discontinues derivative.

28
Typical ?-functionsWhere the Robustness in
Practice Comes From

• 2-norm
• 1-norm
• Huber norm
• Truncated quadratic
• Bi-Squared

• Discards Outliers.
• Smooth.

29
Quantifying RobustnessA peak at tools for
analysis
Bias vs. Variance
30
Quantifying RobustnessA peak at tools for
analysis
Related to variance, on the previous slide
31
Quantifying RobustnessYou want to be robust over
a range of models
32
Quantifying RobustnessA peak at tools for
analysis

• Other measures (Similar)
• Breakage Point How many outliers can an
  estimator handle and still give reasonable
  results.
• Asymptotic bias What bias does an outlier impose.

33
Back to Imageshere we have multiple models
To fit or describe the bulk of a data set well
without being perturbed (influenced to much) by
a small portion of outliers. This should be done
without a pre-processing segmentation of the data.
34
Optimization Methods

• Typical Approach
• Find initial estimate.
• Use Non-linear optimization and/or EM-algorithm.
• NB In this course we have and will seen other
  methods e.g. with guaranteed convergence

35
Hough TransformOne off the oldest robust methods
in visionOften used for initial estimate.
Curse of Dimesionality PROBLEM
Example from MatLab help
36
RanSaCSampling in Hough space, better for higher
dimensions

• RANdom SAmpling
• Consensus, RANSAC
• Iterate
• Draw minimal sample.
• Fit model.
• Evaluate model by Consensus.

• In a Hough setting
• 1. and 2. corresponds to finding a good bin in
  Hough space.
• 3. Corresponds to calculating the value.

Run RanDemo.m
37
RansacHow many iterations
38
Iteratively Reweighted Least Squares IRLSEM-type
or chicken and egg optimization