Pattern recognition system

1
??? ???? ??

• ????? ?????? ????
• ? ? ?

2
?? ??

• ?? ? ?? ? 0900 1200, ????????? ?? 1?
• ?1???
• Required textbook Pattern classification, 2nd
  edition, R. O. Duda, P. E. Hart, D. G. Stork
• References
• Pattern recognition, statistical, structural and
  neural approaches, R. Schalkoff, Wilely
• Statistical digital signal processing and
  modeling, M. H. Hayes, Wiley
• Wavelet transform Introduction to theory and
  application, R. M. Rao, A. S. Bophardica, Wiley
• Selected papers on invasive brain-computer
  interface (BCI) and neural spike sorting (to be
  announced, 5 10 papers)
• Contents
• Fundamentals of statistical pattern recognition
  (Ch. 1, 2, 3, 5, 6, 10 of Duda and Hart)
• Preprocessing of biomedical signal for pattern
  recognition
• Application of pattern recognition system to
  neural spike sorting for the analysis of nervous
  system, brain-computer interface (for neural
  prosthesis)

3
Ch. 1. Introduction
4
Machine perception

• Objective to design and build machines that can
  recognize patterns
• Example
• Speech recognition
• Fingerprint identification
• Character recognition
• DNA sequence identification
• ECG abnormal beat detection

5
Pattern recognition system
A priori information
Sensor
Modeling
Feature extractor
Measurement vector
Feature vector
Sensor
Classifier
Feature extractor
Input data
Decision
6
Ex. Bass or Salmon
7
Ex. Bass or Salmon
No single threshold value of the length will
serve to unambiguously discriminate between the
two categories using length alone, we will have
some errors.
8
Patterns and features

• Objects represented by set of measurements
• Measurement image, waveform
• Pattern
• a set of measurements
• represented by a vector in an n-dimensional space
  (multi-dimensional space, hyperspace)
• represented as a point in hyperspace.
• Feature vector
• Derived from the pattern vector
• Smaller dimension (compared to pattern vector
  dimension)

9
Feature vectors in hyperspace
10
Bass or salmon? using 2-D feature vector
Overall classification error on the data shown is
lower than if we use only one feature as in Fig.
1.3, but there will still be some errors.
11
Decision boundary and generalization

• Overly complex models complicated decision
  boundaries
• Perfect classification of our training samples,
  but poor performance on future patterns.

12
Decision boundary and generalization

• The decision boundary shown might represent the
  optimal tradeoff between performance on the
  training set and simplicity of classifier,
  thereby giving the highest accuracy on new
  patterns.

13
Typical structure of pattern recognition system

• Sensor inputs -gt signal data
• Segmentor isolates sensed object from the
  background
• Feature extractor ????? ??? ??? ??
• Classifier sensed object? ?? class? ??
• Post-processor ?? ?????? (context, cost of
  errors )? ???? ??? action? ??

14
Sensing

• Input to a pattern recognition system is often
  some kind of a transducer (ex. Camera, microphone
  )
• Problem characteristics and limitation of the
  transducer
• ( bandwidth, resolution, sensitivity,
  signal-to-noise ratio )

15
Segmentation and grouping

• Isolate each fish from others on the conveyor
  belt
• In practice, the fish would be abutting or
  overlapping
• Difficult process in practice particularly in
  speech recognition system

16
Feature extraction

• Feature ??? ??? ??? ??? ??? ??? ?? ???? feature
  ???? ?? trivial? ???? ?? (vice versa)
• Traditional goal of the feature extractor to
  characterize an object to be recognized by
  measurements whose values are
• very similar for objects in the same category
• very different for the objects in different
  categories
• Seeking distinguishing features
• In the fish example size of the fish may not be
  important

17
Classification

• Classifier uses feature vector to assign the
  abject to a category
• Abstraction provided by the feature-vector
  representation of the input data ?
• enables the development of domain-independent
  theory of classification
• The degree of difficulty in the classification
  problem
• within-cluster variability vs. between-cluster
  variability
• Variability 1. Problem complexity. 2. Noise

18
Post-processing

• Pattern classifier generally is to be used to
  recommend actions
• put this fish in bucket A, put that fish in
  bucket B
• Post-processor uses the classifier output to
  decide the action
• Measure of classifier performance classification
  error rate, total expected cost (risk)
• Post-processor may be able to exploit context
  T-( )-e, c-( )-t
• Combination of classifier

19
Design of pattern recognition system

• Data collection for the training and testing of
  the system
• Feature selection using domain knowledge and
  experience
• Choose model pattern
• Training determine system parameters
• The results of evaluation may call for
  repetition of various steps in this process in
  order to obtain satisfactory results.

20
Learning and adaptation (Training)

• Learning (training) training data? ???? pattern
  ???? parameter? ??? ??
• Supervised training a teacher provides a
  category label or cost for each pattern in a
  training set
• Unsupervised no explicit teacher, and the system
  forms clusters or natural groupings of input
  patterns

21
Pattern recognition system
A priori information
Sensor
Modeling
Feature extractor
Measurement vector
Feature vector
Sensor
Classifier
Feature extractor
Input data
Decision
22
A good reference paper

• Statistical pattern recognition a reviewJain,
  A.K. Duin, R.P.W. Jianchang MaoIEEE
  Transactions on Pattern Analysis and Machine
  Intelligence,, Volume 22 , Issue 1 , pp.
  4-37,Jan. 2000