Improving Chinese handwriting Recognition by Fusing speech recognition - PowerPoint PPT Presentation

About This Presentation
Title:

Improving Chinese handwriting Recognition by Fusing speech recognition

Description:

Improving Chinese handwriting Recognition by Fusing speech recognition Zhang Xi-Wen CSE, CUHK and HCI Lab., ISCAS 2005.4.12 – PowerPoint PPT presentation

Number of Views:218
Avg rating:3.0/5.0
Slides: 25
Provided by: zxw
Category:

less

Transcript and Presenter's Notes

Title: Improving Chinese handwriting Recognition by Fusing speech recognition


1
Improving Chinese handwriting Recognition by
Fusing speech recognition
  • Zhang Xi-Wen
  • CSE, CUHK and HCI Lab., ISCAS
  • 2005.4.12

2
Outline
  • 1 Chinese handwriting recognition
  • 2 Chinese speech recognition
  • 3 Information fusion
  • 4 Experimental results

3
Handwriting Recognition
  • Handwriting segmentation
  • Character recognition

4
1.1 Handwriting segmentation
  • It is more difficult for Chinese handwriting
    segmentation

5
Character extraction using histogram
  • A histogram of between-stroke gaps.
  • The dimidiate threshold of the histogram is to
    extract lines of strokes.
  • The dimidiate threshold of the histogram of a
    line of strokes is to extract characters.

6
Figure 1. Handwriting segmentation
7
Problems remained
  • A Chinese character may be mis-segmented into
    many characters.
  • Many Chinese characters may be mis-grouped as a
    character.
  • The segmentation error will inevitably result in
    handwriting recognition errors.

8
1.2 Character recognition
  • Isolated character recognizer from HW
  • Many candidates

9
Handwriting.
Text recognized from the handwriting.
The ground-truth text.
Figure 2. Handwriting recognition
10
2 Speech recognition
  • Chinese speech.
  • On-line, microphone.
  • Continuous speech recognizer from MS.

11
Text recognized from the speech corresponding to
the handwriting.
The ground-truth text.
Figure 3. Speech recognition
12
3 Text fusion
  • An optimization problem
  • Dynamic Programming

13
3.1 Principles
  • The fused text should contain more semantic
    information.
  • Construct a text with the least characters and
    the most semantic information.

14
3.2 Four ways
Text recognized from the handwriting.
Text recognized from the speech corresponding to
the handwriting.
Figure 4. Texts to be fused
15
3.3 Dynamic Programming
  • A directed graph.
  • Optimal paths.

16
Figure 5. A directed graph with N levels.
17
(a) Text recognized from the handwriting.
(b) Text recognized from the speech corresponding
to the handwriting.
(c) The optimal fused text corresponding to the
optimal path.
(d) The ground-truth text.
Figure 6. Text fusion using DP.
18
3.4 A language model
  • Lexicon
  • Syntax
  • Semantic

19
Lexicon
20
(No Transcript)
21
4 Experimental results
22
(No Transcript)
23
(No Transcript)
24
  • Thank you very much for
  • your criticism, comments and suggestions!
  • Email xwzhang_at_cse.cuhk.edu.hk
  • Tel 3163-4260
Write a Comment
User Comments (0)
About PowerShow.com