Is%20Question%20Answering%20an%20Acquired%20Skill?

1
Is Question Answering an Acquired Skill?

• Ramakrishnan, Chakrabarti, Paranjpe,
  Bhattacharyya
• Paper presentation Vinay Goel

2
Introduction

• Question Answering (QA) system
• Most QA systems are substantial team efforts
• Difficult to reproduce a well tuned QA system
  from scratch, gauge the benefit of new
  algorithmic ideas, new corpora and new languages
• QA systems
• Complex building blocks like taggers and parsers
• Lashed together with customized glue
• Crucial knobs best preset by QA specialists

3
Goal

• Decompose the QA task cleanly into discovering
  features and learning to score answer snippets
• QA system
• Performs fast, shallow processing of corpus
• Structures the scoring task using features and
  learners
• Trains its scoring algorithm from a past history
  of questions and vetted answers
• Can include side information (Wordnet etc.)
• Reuses expertise accumulated from one corpus to a
  new corpus

4
Noisy simulation perspective

• In a structured database with a suitable schema
  and a structured query language, information
  needs can be expressed clearly
• See QA as a transformation of this process by
  adding natural language from an unknown
  generative process, for both query and data
• Given a question, discover structured fragments
  in it
• Extract selectors which will appear almost
  unchanged in an answer passage
• Extract atype clues, which tell what else to look
  for in a passage that satisfied all selectors

5
Typical connections between a question and answer
6
Atype

• Minimal subclass of entities which will answer a
  question
• Two representations important to factoid QA
• Atype as synset
• Atype as surface patterns

7
Atype as synset

• Q Name an animal that sleeps upright
• A horse
• Wordnet helps recognize that horse is an instance
  of animal
• Most answers which are common nouns are assisted
  by this representation

8
Atype as surface patterns

• Infinite or very large domains such as numbers,
  person names, place names etc. cannot be covered
  by Wordnet
• Logically augment Wordnet to add connections from
  synsets to pattern matchers such as at DDDD or
  Xx said etc.

9
From the question to an atype

• Set of common wh-words
• Questions starting with when, where and who
  immediately reveal their expected atypes
• Word after how is almost always a clue
• Questions using using what and which mention
  atype directly

10
Shallow parsing to extract atype

• Shallow parsing involves finding noun phrases,
  modifiers and attachments between phrases
• Purely based on POS tags
• Strategy for locating atype clues from what and
  which questions
• Head of NP appearing before the auxiliary / main
  verb if it is not a wh-word
• Otherwise, head of NP appearing after the verb

11
Learning to map atype

• When, where, who and how do not directly use a
  term that describes a synset
• Augmented synsets based on surface patterns
  (DDDD) may come handy
• Devised a learning module to help compile
  mappings between short token sequences in
  questions to atypes

12
Selectors

• Second blank in the SQL query selectwhere
• In QA, simply a set of words in the question that
  are expected to appear unchanged in the answer
  passage

13
Identifying selectors

• Choice of features
• POS
• POS assigned to left and right neighbors
• Whether the word starts with an uppercase letter
• Whether the word is a stopword
• Some version of IDF
• How many senses the word has in isolation
• For a given sense, how many other words describe
  the sense

14
How to use selectors

• Two places
• Pad the initial keyword query
• Rerank the candidate phases
• Experiments insist that the response by the word
  search engine (Lucene)
• Contains all selectors
• Use OR over other question words

15
Learning to score passages

• If (q,r) is a positive instance, it is expected
  that
• All selectors match between q and r
• r has an answer zone a which does not contain
  selectors
• The linear distance between a and matched
  selectors in r, tend to be small
• a has strong Wordnet-based similarity with the
  atype of q

16
Overall architecture
17
Experiments

• TREC QA track
• Picked sliding windows of three sentences as
  passages
• Questions and passages were tokenized using GATE
• For learning tasks, used J48 decision tree and
  the logistic regression packages in WEKA

18
Extracting atypes from shallow parses
19
Spotting selectors in questions
20
Passage Reranking performance
21
MRR improvement via reranking
22
Training on corpus of another year
23
Conclusion

• QA system built by wrapping logic around text
  indexers, taggers, shallow parsers and
  classifiers
• Simple assembly of building blocks
• Future work involves improving performance of
  different blocks