Classification of a Speaker Using Support Vector Machines

1

• Classification of a Speaker Using Support Vector
  Machines
• CS679 Term Project
• by Gil-Jin Jang
• Computer Science Department
• KAIST

2
REFERENCES

• Textbook, CHAPTER 6, Support Vector Machines.
• Steve Gunn. 1998. Support Vector Machines for
  Classification and Regression, ISIS Technical
  Report.
• Corinna Cortes, Vladimir Vapnik. 1995. Support
  Vector Networks, Machine Learning 20
  pp.273-297.
• http//www.isis.ecs.soton.ac.uk/, Image Speech
  Intelligent Systems (ISIS) Group, U. Southhampton
• Vladimir Vapnik. 1995. The Nature of Statistical
  Learning Theory, Springer-Verlag.

3
ABSTRACT

• Support Vector Machines
• non-linear classification (and regression)
• constructing hyperplane discriminative, binary
  classifier
• usually faster training than typical NN
• find unique, optimal solution when no
  classification error exists
• Speaker Classification Text-Independent Case
• ill-formed classification problem
• a speaker is characterized by a sequence of
  vectors
• Results
• pairwise binary classification for multiple class
• compared to MLP and RBFN

4
Support Vector Classifier (1)

• Training
• maximizes the distance between 2 classes
• OSH optimal separating hyperplane, (n-1)
  dimension
• support vectors nearest vectors to OSH

Support Vectors
OSH
wxb0
class 1 wxb gt 0 class 2 wxb lt 0
5
Solutions to SVC (1)

• Separable Case
• constraints
• xi training vectors, yi 1, label for each
  class
• yi wxib ? k gt 0, guarantee minimum distance
• min yi wxib 1, unique solution,
  support vectors
• minimize w2 subject to
• yi wxib ? 1
• Non-Separable Case
• providing penalty term ?i
• minimize w2 C ??i subject to
• yi wxib ? 1 - ?i, ?i ? 0

6
Solutions to SVC (2)

• Non-Separable Case Solution
• Lagrange multipliers and the Kuhn-Tucker Theorem
• equivalent dual problem (detail omitted)
• minimize W(?, x, y) subject to
• 0 ? ?i ? C, ??i yi 0
• w ? support vectors ?i yi xi
• classification
• f(x) sign( wx b )
• sign( ??i yi (xix)).

7
Expansion to non-linear case

• Expand to Higher-Dimension Space
• ax by c
• gt ax2 by2 cxy dx ey f
• (direct computation is intractable)
• Any Kernel Function
• replace inner product with some kernel function
• (X, Y) Kd(x, y) (x y 1)d
  (polynomial)
• Gaussian/exponential RBF, tanh, spline, etc.
• classification
• f(x) sign( ??i yi (X, Y) ).

8
Speaker Classification

• Sequence of Short-Time Analysis Vectors
  (Cepstrum)
• Classified by Sum of the Corresponding Outputs
• TOTAL_SCORE( X, Class ) ? score( xi, Class )
• select argmax Class TOTAL_SCORE( X, Class )
• Classification by SVC
• using score( xi, Class ) as
• f(x) sign( wx b ).
• f(x) wx b . distance to
  hyperplane
• multiple class classification (with binary
  classifiers)
• g(x, Class i) sum i?j fij(x)

9
Classification Examples (1)
polynomial, p 2. only two components are
selected.
10
Classification Examples (2)
polynomial, p 3. only two components are
selected.
11
Experiments

• Configuration
• 10 speakers, Text-Independent
• Features 15-dimensional cepstrum vectors
• Training Data about 15 seconds (1500 vectors)
• Testing Data about 5 seconds (500 vectors),
  excluded from training data.
• MLP(16 x 20 x 10), Gaussian RBFN(15 x 10 x 10)
• SVM
• only polynomial Kernel functions were considered.
  (p2, p3)
• 2 kinds of classification functions ( f(x), f(x)
  )

12
Experimental Results
Neural Networks
Support Vector Machines
13
Analysis Conclusion

• SVM applied to Speaker Classification Problem
• Better or Equal Classification Power with
  MLP/RBFN
• find unique and optimal solution with given
  constraints
• faster training, slower classification than NN.
• polynomial degree 3 maybe overfitted
• classification function f(x)
• calculate cost function as distance to hyperplane
• sensitive to outliers
• More on SVM
• Selection of Kernel Function
• Design of Multiple Classification Strategy