Combining Neural Networks and Context-Driven Search for On-Line, Printed Handwriting Recognition in the Newton

1
Combining Neural Networks and Context-Driven
Search for On-Line, Printed Handwriting
Recognition in the Newton

• Larry S. Yaeger, Brandn J. Web, and Richard F.
  Lyon

2
Overview
Words
(x,y) Pen-lifts
Tentative Segmentation
Context Search
Character Segmentation Hypotheses
Neural Net Classifier
Character Class Hypotheses
3
Classifier

• Backpropogation Network
• Multiple Inputs
• 20 x 9 Stroke Feature
• 14 x 14 Image
• 5 x 1 Stroke Count
• 1 x 1 Aspect Ratio
• All run in parallel and combined at output layer.

4
Picking the Top Choices

• Ambiguity in character recognition
• Problem if the full error is propagated back,
  the network gives one high answer, but no
  secondary answers.
• Solution Normalize the error before propagating
  back.

5
Ambiguity Example
6
Ambiguity Example
ASS
7
Ambiguity Example
uh
8
Removing Ambiguity

• Use the context
• Look up possible words in a dictionary
• Use word spacing to help
• Could use sentence structure, but this doesnt

9
LVQ Selection of Networks

• An LVQ Network will be used to select an
  appropriate BP network for a given context.
• EX Driving in heavy traffic on the freeway vs.
  driving in town.

10
Reasons for Combining Networks

• Allows for a separation of tasks
• A BP Network must be carefully trained to adapt
  to different situations
• It would be very difficult to add a different
  subtask to a BP network.
• Hope that this technique will scale to more
  complex tasks better.

11
Training

• Manual Construction
• Train BP networks for sub-tasks separately, and
  then train the LVQ network to select the correct
  network.
• Two-Stage Construction
• Use the LVQ to classify all of the training set
  into different classes. Train a BP network for
  each of these classes.

12
Training (cont.)

• Simultaneous Construction
• Instead of using the standard distance measures,
  use the output of the BP networks as the distance
  measure. For each case, the LVQ network will be
  adjusted so that the winner moves closer to the
  result. Each case that passes through a BP
  network will be used to adjust the weights of
  that network.

13
Project Goals

• Create the network described and implement all
  three training methods
• Compare the results to those obtained from a
  single large BP network.