Comparison of handwritings 2' Specification

1
Comparison of handwritings2. Specification
Architecture

• Miroslava Boeková
• Thesis supervisor Doc. RNDr. Milan Ftácnik, PhD
• http//sprite.edi.fmph.uniba.sk/bozekova

2
Goal

• Input is one or more handwritings
• the task is to create methods, which could
  determine whether the document/documents was/were
  written by the same person or not.
• Implement and make experiments.

3
I / O data

• Input scanned images
• Output data information

4
Input

• IAM Handwriting Database
• My own scanning images
• Another Databases
• For more information see the first presentation
  Experimental Data Sets

5
Application I.

• Preprocessing
• Grayscale
• Binarization
• Normalization deskew document
• Noise reduction (identification handwriting from
  noisy documents)
• Rule Line Removal
• Lines Segmentation

6
Application II.

• Normalization deskew and deslant
• Words Segmentation
• Graphemes Segmentation
• Grapheme codebook generation
• Feature extraction
• Feature matching/combination
• Writer verification
• Forgery detection

7
Binarization I.
8
Binarization II.

• Thresholding techniques
• Otsus method 1979 (global method)
• Kapur 1985, Niblack 1986
• Wang and Pavlidis 1993, Brink 1995
• Solihin and C.Leedham 1999 (two global
  techniques native integral ratio (NIR) and
  quadratic integral ratio (QIR))
• Sauvola 2000
• Zhang and Tan 2001 (improved Niblack)

9
Normalization I.
10
Normalization II.

• E. Kavallieratou, N. Fakotakis, G. Kokkinakis.
  2002. Skew angle estimation for printed and
  handwritten documents using the WignerVille
  distribution. Image and Vision Computing 20
  (2002) 813824. 2002.

11
Identification handwriting I.
12
Identification handwriting II.

• Segmentation process divide into regions - a
  rectangular window (size determined dynamically
  for each document or size of the character, word,
  zone)
• Classification process identify regions as one
  of Machine-print, Handwriting or Noise
• Noise - salt and pepper noise, scan noise,
  scratches, black borders, logos

13
Identification handwriting III.

• Y. Zheng. 2006. Handwriting identification,
  matching and indexing in noisy document images.
  LAMP-TR-129/CS-TR-4781/UMIACS-TR-2006-06,
  University of Maryland, College Park, January
  2006. 2006.
• S. Shetty, H. Srinivasan, M. Beal and S. Srihari.
  2007. Segmentation and labeling of documents
  using Conditional Random Fields. Document
  Recognition and Retrieval XIV, Xiaofan Lin
  Berrin A. Yanikoglu, Editors, 65000U. 2007.

14
Rule Line Detection and Removal I.
15
Rule Line Detection and Removal II.

• Directional singly-connected chains (DSCC)

16
Rule Line Detection and Removal III.

• DSCC-based text filtering
• perform horizontal vertical projection with a
  hidden Markov model (HMM) decoding process to
  detect lines
• The Viterbi algorithm - search the optimal
  positions of lines simultaneously from the
  projection profile.
• Line removal algorithm - a line width
  thresholding based approach

17
Rule Line Detection and Removal IV.
18
Rule Line Detection and Removal V.

• Y. Zheng. 2006. Handwriting identification,
  matching and indexing in noisy document images.
  LAMP-TR-129/CS-TR-4781/UMIACS-TR-2006-06,
  University of Maryland, College Park, January
  2006. 2006.

19
Line Segmentation I.
20
Line Segmentation II.

• Chain code document representation
• Get initial set of candidate lines.
• Decisions if obstructed components belong above
  or below (probabilistic, distance)

21
Line Segmentation III.
22
Line Segmentation IV.

• M. Arivazhagan, H. Srinivasan and S. Srihari.
  2007. A Statistical approach to line segmentation
  in handwritten documents. Document Recognition
  and Retrieval XIV, Xiaofan Lin Berrin A.
  Yanikoglu, Editors, 65000T. 2007.

23
Deskew and deslant lines
24
Word segmentation I.

• Several approaches
• the gaps between words (inter-word gaps) are
  larger than those inside the words (intra-word
  gaps) connected components the distances
  between components (using some heuristic called
  gap metric) a gap is an interword gap if the
  size of the gap is above a threshold value the
  extracted words may contain punctuation marks
  (e.g. dot, comma, etc.) attached

25
Word segmentation II.

• Using a neural network
• Scale space techniques
• The utilization of semantic knowledge
• Structure tree

26
Word segmentation III.
27
Word segmentation IV.
28
Word segmentation III.

• Tamás Varga. 2006. Off-line Cursive Handwriting
  Recognition Using Synthetic Training Data.
  Medium Paperback, Year of Publication 2006,
  ISBN158603636X. 2006.

29
Graphemes segmentation I.

• Grapheme (sub or supra-allographic fragments) -
  may or may not overlap a complete character
• Allograph - one particular letter from an
  alphabet can be realized using a number of
  shapes.
• Ligatures - the line segments that form
  connections between characters

30
Graphemes segmentation II.

• segment the ink at the minima in the lower
  contour with the condition that the distance to
  the upper contour is on the order of the
  ink-trace width.

31
Graphemes segmentation III.

• graphemes are extracted as connected components,
  followed by a size normalization to 30x30 pixel
  bitmaps, preserving the aspect ratio of the
  original pattern.

32
Grapheme codebook generation I.

• 3 clustering methods will be used to generate the
  grapheme codebook
• k-means (partitional clustering number of
  clusters (k) is dedicated in advance)
• Kohonen SOM 1D and 2D (self-organizing map,
  without teacher).

33
Grapheme codebook generation II.
34
Grapheme codebook generation III.
35
Grapheme codebook generation IV.
36
Grapheme codebook generation II.

• Kohonen training show spatial order
• k-means is disorderly (k number of clusters)
• ksom1D - a long linear string of shapes and
  gradual transitions can be observed if the map is
  read in left-to-right top-to-bottom order.
• The ksom2D codebook shows a clear bidimensional
  organization.

37
Writer verification vs. Writer identification
38
Writer verification I.

• is to determine whether two documents were
  written by the same person or not
• ideal world no forged or disguised handwriting
• 2 levels the texture level (habitual pen-grip -
  slant) and the allograph (character-shape) level.
  

39
Writer verification II.

• feature extraction
• probability distribution functions (PDFs) -
  vector of probabilities capturing a facet of
  handwriting uniqueness
• feature matching/feature combination
• writer verification

40
Writer verification III.
41
Writer verification IV.

• Contour-Direction PDF (f1)

42
Writer verification V.

• Contour-Hinge PDF (f2)

43
Writer verification VI.

• Direction Co-Occurrence PDFs (f3h, f3v)

44
Writer verification VII.
45
Writer verification VIII.

• the PDF features (f1, f2, f3, f4, f5)
• distance measure between the feature vectors
  between two given handwriting samples
• q, i samples
• k is the number of bins
• in the PDF (the dimensionality of the feature),
  p are entries in the PDF

46
Writer verification IX.

• other distance measures Hamming, Euclid,
  Bhattacharya, Minkowski up to order 5, 3,2 and
  Hausdorff.
• Distances up to a predefined decision threshold T
  are deemed sufficiently low for considering that
  the two samples have been written by the same
  person.
• Beyond T, the samples are considered to have been
  written by different persons.

47
Writer verification X.
48
Forgery detection I.

• Sung-Hyuk Cha, Yi-Min Chee, and Charles C.
  Tappert. 2004. Automatic Detection of Handwriting
  Forgery Using a Fractal Number Estimate of
  Wrinkliness. International Journal of Pattern
  Recognition and Artificial Intelligence, Vol. 18,
  No. 7 (2004) 1361-1371. 2004.

49
Forgery detection II.

• forgers often carefully copy or trace genuine
  handwriting
• good forgeries retain the shape and size of
  genuine writing are usually written more slowly
  and are therefore wrinklier (less smooth) than
  genuine writing.
• the wrinkliness of the good forgeries is
  significantly greater than that of the genuine
  writings

50
Forgery detection III.

• This wrinkliness feature can be measured using a
  fractal dimension measure.
• 8 features
• Centroid ratio
• Slant, Stroke width, and Ascender/Descender
• side-projected histogram, bottom-projected
  histograms, the gradient histogram
• the wrinkliness

51
Forgery detection IV.
52
Forgery detection V.
53
Forgery detection VI.
54
Three layer architecture
55
GUI

• Work environment
• Graphs
• Results of methods/algorithms

56
Application
57
Database

• Storing the following data
• Information about images (author, paper, type of
  writing utensil, source,) XML
• Programs results (which images, which
  methods/algorithms, errors,)
• Will be input for experimental research

58
Software

• Java
• NetBeans vs. Eclipse
• Libraries
• GUI Swing, Awt
• Jimi - class library for managing images (GIF,
  JPEG, TIFF, PNG, PICT, Photoshop, BMP, )
• JAXB, JAXP
• MySQL, XML

59
References

• TO DO

60

• The End
• May 2007