Negative Results and Open Problems - PowerPoint PPT Presentation

1 / 9
About This Presentation
Title:

Negative Results and Open Problems

Description:

A wealth of knowledge is lost. ... 1967 - Cover-Hart: Nearest neighbors. 1982 - Hopfield nets, distributed associative memories. ... – PowerPoint PPT presentation

Number of Views:29
Avg rating:3.0/5.0
Slides: 10
Provided by: Isabell47
Category:

less

Transcript and Presenter's Notes

Title: Negative Results and Open Problems


1
Negative Results and Open Problems
NIPS 2002 workshops
  • Isabelle Guyon
  • Clopinet

2
Why?
  • Negative results are seldom reported. A wealth of
    knowledge is lost.
  • Negative results are sometimes more informative
    that positive results.
  • Negative results sometimes point to important
    open problems.

3
History
  • We better pay attention to negative results
    neural networks have been killed several times
    by negative results.

time
SVMs (1992)
Minski Papert (1969)
4
Negative prejudices
  • 1) Against multilayer networks
  • 1949 - MinskyPapert XOR problem.
  • 1983 - Kirkpatrick et al simulated annealing.
  • 1985 - HintonSejnowski Boltzmann Machine.
  • 1986 Backprop.

5
Negative prejudices
  • 2) Against polynomials
  • 1975 - T. Poggio kernel trick for polynomials.
  • 1977 - J. Schürmann feature selection.
  • 1984 -T. Kohonen popularizes Poggios results.
  • Lost popularity when back-prop arrived, 1986.
  • Become an example of overfitting model.
  • 1992 - SVMs Regain of interest.

6
Negative prejudices
  • 3) Against kernel methods
  • 1964 - Aizerman et al Potential functions.
  • 1967 - Cover-Hart Nearest neighbors.
  • 1982 - Hopfield nets, distributed associative
    memories. Parody of the Grand-Mother cell
    methods.
  • 1992 - SVMs Regain of interest.

7
Negative prejudices
  • 4) Against biased estimators
  • 1922 - Fisher promotion of the use of unbiased
    estimators in statistics.
  • 1971 - Vapnik-Chervonenkis theory biased
    estimators are good to get better
    generalization.
  • (this example is a courtesy of V. Vapnik)

8
Other prejudices
  • Against greedy search.
  • For multivariate feature selection.
  • For small VC dimension.
  • For sparse solutions.
  • For introducing domain knowledge.
  • Your negative result here.

9
Some of my negative results
  • SVM clustering does not work in high dimensions.
  • Feature selection with correlation methods often
    works better than multivariate methods.
  • Dumb linear, poly2 and Gaussian kernels are hard
    to beat.
  • Hard to say of MSE and SVM which is best to train
    kernel classifiers.
Write a Comment
User Comments (0)
About PowerShow.com