Hindi POS tagging and chunking : An MEMM approach

1
Hindi POS tagging and chunking An MEMM approach

• Aniket Dalal
• Kumar Nagaraj
• Uma Sawant
• Sandeep Shelke
• Under the guidance of Prof. P. Bhattacharyya

2
Introduction

• Lexical Analysis
• Part-Of-Speech (POS) Tagging Assigning
  part-of-speech to each word. eg. Noun, Verb...
• Syntactic Analysis
• Chunking Identify and label phrases as verb
  phrase, noun phrase etc.

3
Outline

• Maximum Entropy Markov Model (MEMM)
• Principle
• Mathematical formulation
• System overview
• Parameter estimation and classification
• POS tagging features
• Chunking features
• Results and error analysis
• Future work
• Conclusion

4
Maximum Entropy Markov Model

• Maximum entropy principle
• The least biased model which considers all known
  information is the one which maximizes entropy.
• Mathematical formulation
• Maximize entropy

5
Maximum Entropy Markov Model

• Mathematical formulation...
• Under constraints

The distribution with the maximum entropy(p) is
equivalent to
6
System overview

• Parameter estimation and classification
• GIS (Generalized Iterative Scaling)
• finds the model parameters that define the
  maximum entropy classifier for a given feature
  set and training corpus
• Beam Search
• heuristic search algorithm, optimization of
  best-first search
• unfolds the first m most promising nodes at each
  depth

7
What are features?

• Feature function
• A boolean function which captures aspects of the
  language
• For example,

8
POS tagging features

• Context-based
• POS tag of previous word
• Current word
• Word-dependent
• Suffixes, digits, special characters, English
  words

9
POS tagging features

• Dictionary-based
• Possible tags for the word, according to the
  dictionary
• Corpus-driven
• Occurrence of a word and its tag(s) according to
  the training data

10
Chunking features

• Context based features
• Word itself
• POS tag
• Nonessential-word tag
• Chunk label
• Current POS tag based feature
• Tag class

11
Results

• POS tagging accuracy
• Best 89.346
• Average 88.4
• Chunk labeling accuracy (per word basis)
• Best 87.399
• Average 86.45

12
Accuracy across runs
13
Error Analysis

• Good performance for
• VAUX, VFM, VNN
• Need to improve
• Compound nouns
• Proper nouns

14
Future Work

• Morphological Features
• Enriching dictionary
• Hybrid models

15
References

• Adwait Ratnaparakhi. 1996. A maximum entropy
  model for part-of-speech tagging. In Erich Brill
  and Kenneth Church, editors, Proceedings of the
  Conference on Empirical Methods in NLP, pages
  133-142. ACL. Somerset, New Jersey.
• Adwait Ratnaparakhi. 1997. A simple introduction
  to maximum entropy models for natural language
  processing. Technical report 97-08, Institute for
  Research in Cognitive Science, University of
  Pennsylvania.

16
References

• Adam L. Berger , Vincent J. Della Pietra ,
  Stephen A. Della Pietra, 1996 .A maximum entropy
  approach to natural language processing,
  Computational Linguistics, v.22 n.1, p.39-71.
• Akshay Singh, Sushma Bendre, and Rajeev Sangal.
  2005. HMM based chunker for hindi. In Proceedings
  of IJCNLP-05. Jeju Island, Republic of Korea.

17
References

• J N Darroch, D Ratcliff, 1972. Generalized
  Iterative Scaling for Log-Linear Models, The
  Annals of Mathematical Statistics.

18
Thank you!Questions ?