Loop Investigation for Cursive Handwriting Processing and Recognition

1
Loop Investigation for Cursive Handwriting
Processing and Recognition

• By Tal Steinherz
• Advanced Seminar (Spring 05)

2
Outline

• Background on cursive handwriting

• Introduction to loops

• Pattern recognition and machine learning conflicts

• Feature extraction solutions

• Demonstrations and experimental results

3
Cursive Handwriting (J. C. Simon)

• Displacing a pen from left to right in an
  oscillating movement, with loops, descendants
  (legs), and ascendants (poles).

4
Cursive vs. Character

• Cursive continuous concatenated set of
  strokes.produced by a human being in a free
  style.
• Character a single standalone symbol.produced
  by a machine subjected to numerous alternative
  fonts.

5
Online vs. Offline

• Online captured by pen-like devices.the input
  format is a two-dimensional signal of pixel
  locations as a function of time (x(t),y(t)).
• Offline captured by scanning devices.the input
  format is a two-dimensional image of gray-scale
  colors as a function of location I(mn).strokes
  have significant width.

6
Online vs. Offline (demo)
7
A Loop (T. Steinherz)

• A set of neighboring foreground pixels
  surrounding a hole, i.e., a connected blocked
  group of background pixels in the words image,
  where all foreground pixels are within stroke
  width distance from the hole.

8
Ascending (Descending) Loops
9
Axial (of the middle zone) Loops
10
The importance of loops

• Shared by many letters (especially a,d,e,g,o,p,q)
• Byproduct of the continuous nature of cursive
  handwriting (like with b,f,h,j,k,l,s,t,y,z)
• Elementary and prominent features
• Carry additional information given by a set of
  descriptive parameters

11
The motivation to investigate loops

• Character recognitionsupports discrimination
  between letters.
• Writer modeling
• Identification
• Examination
• contributes to applications in forensic science
  and graphology.

12
The output of loop investigation

• Incomplete (open) loop identification
• Hidden (collapsed) loop tracking - locating blobs
  that correspond to online loops
• Multi (encapsulated) loops understanding -
  distinguishing natural from artificial loops
• Temporal information recovery - retracing the
  original path of a pen

13
The Engineering Approach(J. C. Simon T.
Pavlidis)

• Requires understanding the structure of the
  objects to be recognized and apply the
  appropriate combination of (pattern recognition)
  techniques.

14
Feature extraction dilemmas

• Offline cursive word signal representation
• Loop identification
• Signal to noise ratio
• Feature vector translation
• The difficulties consist in the feature
  extraction and preprocessing rather than the
  machine learning \ recognition engine phase.

15
Offline cursive word signal representation

• We use the external upper and lower contours in
  conjunction with the internal contour of all
  visible loops.

16
Loop identification

• Given a set of singular points, identification is
  provided by correlation between pieces of the
  same contour (around anchor points), of the
  opposite contours and\or in association with
  subsets of internal contours.

17
Signal to noise ratio

• In order to improve the signals parametric
  quantifiability and reduce noisy artifacts, the
  contour is transformed to a polygon.

18
Hidden loop tracking -the mutual distance
principle
19
Hidden loop tracking -the mutual distance
principle (cont.)
20
Hidden loop tracking -the mutual distance
principle (cont.)
21
Multi loops understanding -the continuity
principle
22
Temporal information recovery -the matching
principle
23
Hidden loop tracking -an application to
ascending (descending) loops
Experimental Results
24
Hidden loop tracking -an application to
ascending (descending) loops (cont.)
Experimental Results
25
Hidden loop tracking -an application to
ascending (descending) loops (cont.)
Experimental Results
26
Hidden loop tracking -an application to
ascending (descending) loops (cont.)
Experimental Results
Small Loops
Total
Threshold
No Loops
8
180
209
389
209
340
6
131
27
Multi loops understanding -a classifier of
beginning a-s
Experimental Results
More than 40 writers with 1-4 samples per writer.
28
Multi loops understanding -a classifier of
beginning a-s
Experimental Results