Learning piecewise linear classifiers via Dirichlet Process

1
Learning piecewise linear classifiers via
Dirichlet Process
Ya Xue and Xuejun Liao
22 April 2005
2
Piecewise linear classifier

• A piecewise linear classifier is a classifier
  that generates a piecewise linear decision
  boundary.

• Consider the binary problem where the labeled
  data

• We assume that each Dj contains at least one
  positive sample and one negative sample.
• Associated with each Dj there is a linear
  classifier parameterized by wj
• Assume Dj is primitive and the samples in it are
  always linearly separable

3
Piecewise linear classifier (contd)

• It turns out that many of wj will become
  identical to each other, yielding a small number
  of distinct representative ws.
• Each distinct w represents a linear boundary
  that separate a subset of the samples in D and
  the ensemble of w gives us the piecewise linear
  boundary.
• We use Dirichlet Process to learn wij , and
  automatically discover the piecewise linear
  boundary from wij

4
Gibbs Sampling of the Posterior Dirichlet Process
5
Results I

• 1000 burn-ins and 1000 samples.

6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
Results II
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Results M-Logit-DP

• 200 primary data and 200 auxiliary data.
• 5 labeled primary data.
• MCMC 3000 burn-ins and 1000 samples.

14
(No Transcript)
15
(No Transcript)