Introduction to Computational Linguistics

1
Introduction to Computational Linguistics

• Eleni Miltsakaki
• AUTH
• Fall 2005-Lecture 6

2
Whats the plan for today?

• Peer-to-peer tutorial on
• Computational linguistics
• Grammars and parsing
• TAG
• LFG
• HPSG
• Questions about homework
• On-line processing of syntactic ambiguity in
  adults and children

3
Slides to guide you review tutorial
4
What is computational linguistics?

• A discipline between Linguistics and Computer
  Science
• Concerned with the computational aspects of human
  language processing
• Has theoretical and applied components (explain)

5
Why is language hard for computers?

• AMBIGUITY! (GIVE EXAMPLES OF SYNTACTIC/SEMANTIC
  etc AMBIGUITIES)
• Natural languages are massively ambiguous at all
  levels of processing (but humans dont even
  notice)
• To resolve ambiguity, humans employ not only a
  detailed knowledge of the language -- sounds,
  phonological rules, grammar, lexicon etc -- but
  also
• Detailed knowledge of the world (e.g. knowing
  that apples can have bruises but not smiles, or
  that snow falls but London does not).
• The ability to follow a 'story', by connecting up
  sentences to form a continuous whole, inferring
  missing parts.
• The ability to infer what a speaker meant, even
  if he/she did not actually say it.
• It is these factors that make NLs so difficult to
  process by computer -- but therefore so
  fascinating to study.

6
Grammars and parsing

• What is syntactic parsing
• Determining the syntactic structure of a sentence
• Basic steps
• Identify sentence boundaries
• Identify what part of speech is each word
• Identify syntactic relations
• Tree representation
• John ate the pizza
• (S (NP (N John))
• (VP (V ate)
• (NP (Det the)
• (N cat))))

7
How to construct a tree

• To construct a tree of an English sentence you
  need to know which structure are legal in English
• Rewrite rules
• Describe what tree structures are allowed in the
  language
• NPgt N
• NPgt Det NP
• VPgt V
• VP gt V NP
• S gt NP VP
• S
• gt NP VP
• gt N VP
• gt John VP
• gt John V NP
• gt John ate NP
• gt John ate Det N
• gt John ate the N
• gt John ate the pizza

8
Chomskys Hierarchy

• Containment hierarchy of classes of formal
  grammars that generate formal languages
• Type 0 unrestricted, include all formal grammars
• Any string of terminals and non-terminals to any
  string of terminals and non-terminals
• Type 1 context sensitive
• A? any string of terminals and non-terminals
• Type 2 context free (the theoretical basis for
  the syntax of most programming languages)
• A? a, A? Ba
• Type 3 regular grammars
• A ? a

9
Tree adjoining grammar

• Introduced by Joshi, Levy Takahashi (1975) and
  Joshi (1985)
• Linguistically motivated
• Tree generating grammar (generates tree
  structures not just strings)
• Example I want him to leave, I promised him to
  leave
• Allows factoring recursion from the statement of
  linguistic constraints (dependencies), thus
  simplifying linguistic description (Kroch Joshi
  1985)
• Formally motivated
• A (new) class of grammars that describe mildly
  context sensitive languages (Joshi et al 1991)

10
TAG formalism

• Concepts lexicalization and locality/recursion
• Who do you like t?
• Who does John think that you like t?
• Who does John think that Mary said that you like
  t?
• Elementary objects initial trees and auxiliary
  trees
• Operations substitution and adjunction
• Adjunction

11
(No Transcript)
12
Adjunction
13
Adjunction
14
Derived and derivation trees
15
Lexical Functional Grammar

• First introduced by Kaplan Bresnan (1982)
• Two parallel levels of syntactic representation
• Constituent structure (c-structure)
• Functional structure (f-structure)
• C-structures have the form of context-free phrase
  structure trees
• F-structures are sets of pairs of attributes and
  values attributes may be features, such as tense
  and gender, or functions, such as subject and
  object.

16
LFG example
17
Head-driven Phrase Structure Grammar

• aka HPSG
• HPSG home http//hpsg.stanford.edu/

18
(No Transcript)
19
Feature structure in HPSG

• A feature structure is a set of pairs of the form
  ATTRIBUTE value
• ATTRIBUTE an element of the set of features
  named ATT in the grammar (e.g., case, person etc)
• value can be atomic (a string) or another
  feature structure

20
Examples of feature structures
21
Feature types

• Feature structures are of a certain type, written
  in italics
• Features are organized in hierarchies

22
Valence and grammar rules

• Complements are specified as complex categories
  in the lexical representation
• There are also specific rules for head complement
  combinations

23
Representation of valence in feature descriptions
A lexical entry consists of
24
Head feature principle

• In a headed structure, the head features of the
  mother are identical to the head features of the
  head daughter

25
Linguistic generalizations in the type hierarchy

• Types are arranged in a hierarchy
• The most general type is at the top
• Information about properties of an object of a
  certain type are specified in the definition of
  the type
• Subtypes inherit these properties
• Like an encyclopedic entry
• The upper part of the hierarchy is relevant to
  all languages (universal grammar)
• More specific types maybe specific for classes of
  languages or just one language

26
A simple example
27
END OF REVIEW SLIDES
28
Todays question

• How do humans (adults and children) process
  syntactic ambiguity?

29
Trueswell et al 1999

• The kindergarten-path effect Studying on line
  sentence processing in young children, in
  Cognition (1999)

30
The garden-path theory

• At points of syntactic ambiguity the
  syntactically simplest alternative is chosen
  e.g. minimal attachment
• (e.g., Frazier and Rayner 1982, Ferreira and
  Clifton 1986)
• However, it has been shown that non-syntactic
  sources of information can mediate garden-path
  effects
• (e.g., Altmann and Steedman 1988, Tanenhaus
  et al 1995)

31
Referential principle

• Example if two thieves are evoked in the context
  and then we hear
• Ann hit the thief with
• we prefer the NP-attachment reading
• (Crain Steedman 1985)

32
Experiment 1

• Methodology eye-tracking
• Participants 16 5-year-old children
• Material
• Put the frog on the napkin in the box (ambiguous
  between DESTINATION and MODIFIER)
• Put the frog thats on the napkin in the box
  (unambiguous)

33
Head mounted eye tracker
34
1 and 2 referent context
35
Unambiguous
36
Analysis

• Percentage of trials with eye-fixation to
  INCORRECT DESTINATION (I.e. the empty napkin)

37
(No Transcript)
38
Results

• VP-attachment preference for children 5-year
  olds prefer to interpret the ambiguous on the
  napkin as destination regardless of referential
  context
• Children are insensitive to the Referential
  Principle
• They dont recover from initial interpretation
• In the 2-referent ambiguous condition they picked
  the Target animal at chance

39
Experiment 2

• Participants 12 adults
• Same material
• Same methodology

40
(No Transcript)
41
Results

• Adults experienced garden path in the 1-referent
  ambiguous condition only

42
(No Transcript)
43
Conclusions

• Adults and children differ in how they handle
  temporary syntactic ambiguity
• Adults resolve ambiguity according to the
  Referential Principle modifier in 2-referent
  context, destination in 1-referent context
• Children are insensitive to the Referential
  Principle They resolve the ambiguity to the
  VP-attachment interpretation, i.e., destination

44
Explanation of VP-attachment preference in
children

• Minimal attachment?
• Lexical frequency?