Representing Meaning

1
Representing Meaning

• Lecture 19
• 13 Sep 2007

2
Semantic Analysis

• Semantic analysis is the process of taking in
  some linguistic input and assigning a meaning
  representation to it.
• There a lot of different ways to do this that
  make more or less (or no) use of syntax
• Were going to start with the idea that syntax
  does matter
• The compositional rule-to-rule approach
• Compositional Semantics
• Syntax-driven methods of assigning semantics to
  sentences

3
Semantic Processing

• Were going to discuss 2 ways to attack this
  problem (just as we did with parsing)
• Theres the theoretically motivated correct and
  complete approach
• Computational/Compositional SemanticsCreate a
  FOL representation that accounts for all the
  entities, roles and relations present in a
  sentence.
• And there are practical approaches that have some
  hope of being useful and successful.
• Information extractionDo a superficial analysis
  that pulls out only the entities, relations and
  roles that are of interest to the consuming
  application.

4
Compositional Analysis

• Principle of Compositionality
• The meaning of a whole is derived from the
  meanings of the parts
• What parts?
• The constituents of the syntactic parse of the
  input
• What could it mean for a part to have a meaning?

5
Better

• Turns out this representation isnt quite as
  useful as it could be.
• Giving(John, Mary, Book)
• Better would be one where the roles or cases
  are separated out. E.g., consider
• Note essentially GiverAgent, GivenTheme,
  GiveeTo-Poss

6
Predicates

• The notion of a predicate just got more
  complicated
• In this example, think of the verb/VP providing a
  template like the following
• The semantics of the NPs and the PPs in the
  sentence plug into the slots provided in the
  template

7
Advantages

• Can have variable number of arguments associated
  with an event events have many roles and fillers
  can be glued on as appear in the input.
• Specifies categories (e.g., book) so that we can
  make assertions about categories themselves as
  well as their instances. E.g., Isa(MobyDick,
  Novel), AKO(Novel, Book).
• Reifies events so that they can be quantified and
  related to other events and objects via sets of
  defined relations.
• Can see logical connections between closely
  related examples without the need for meaning
  postulates.

8
Example

• AyCaramba serves meat

9
Compositional Analysis
10
Augmented Rules

• Well accomplish this by attaching semantic
  formation rules to our syntactic CFG rules
• Abstractly
• This should be read as the semantics we attach to
  A can be computed from some function applied to
  the semantics of As parts.

11
Example

• Easy parts
• NP -gt PropNoun
• NP -gt MassNoun
• PropNoun -gt AyCaramba
• MassMoun -gt meat

• Attachments
• PropNoun.sem
• MassNoun.sem
• AyCaramba
• MEAT

12
Example

• S -gt NP VP
• VP -gt Verb NP
• Verb -gt serves

• VP.sem(NP.sem)
• Verb.sem(NP.sem)
• ???

13
Lambda Forms

• A simple addition to FOPC
• Take a FOPC sentence with variables in it that
  are to be bound.
• Allow those variables to be bound by treating the
  lambda form as a function with formal arguments

14
Example
15
Example
16
Example
17
Example
18
Syntax/Semantics Interface Two Philosophies

• Let the syntax do what syntax does well and dont
  expect it to know much about meaning
• In this approach, the lexical entrys semantic
  attachments do all the work
• Assume the syntax does know something about
  meaning
• Here the grammar gets complicated and the lexicon
  simpler (constructional approach)

19
Example

• Mary freebled John the nim.

• Who has it?

• Where did he get it from?

• Why?

20
Example

• Consider the attachments for the VPs
• VP -gt Verb NP NP rule (gave Mary a book)
• VP -gt Verb NP PP (gave a book to Mary)
• Assume the meaning representations should be the
  same for both. Under the lexicon-heavy scheme,
  the VP attachments are
• VP.Sem (NP.Sem, NP.Sem)
• VP.Sem (NP.Sem, PP.Sem)

21
Example

• Under a syntax-heavy scheme we might want to do
  something like
• VP -gt V NP NP
• V.sem Recip(NP1.sem)
  Object(NP2.sem)
• VP -gt V NP PP
• V.Sem Recip(PP.Sem) Object(NP1.sem)
• i.e the verb only contributes the predicate, the
  grammar knows the roles.

22
Integration

• Two basic approaches
• Integrate semantic analysis into the parser
  (assign meaning representations as constituents
  are completed)
• Pipeline assign meaning representations to
  complete trees only after theyre completed

23
Semantic Augmentation to CFG Rules

• CFG Rules are attached with semantic attachments.
• These semantic attachments specify how to compute
  the meaning representation of a construction from
  the meanings of its constituent parts.
• A CFG rule with semantic attachment will be as
  follows
• A ? ?1,,?n f(?j.sem,,?k.sem)
• The meaning representation of A, A.sem, will be
  calculated by applying the function f to the
  semantic representations of some constituents.

24
Naïve Approach

• ProperNoun ? Anarkali Anarkali
• MassNoun ? meat Meat
• NP ? ProperNoun ProperNoun.sem
• NP ? MassNoun MassNoun.sem
• Verb ? serves ?e,x,y ISA(e,Serving) ?
  Server(e,x) ? Served(e,y)
• But we cannot propagate this representation to
  upper levels.

25
Using Lambda Notations

• ProperNoun ? Anarkali Anarkali
• MassNoun ? meat Meat
• NP ? ProperNoun ProperNoun.sem
• NP ? MassNoun MassNoun.sem
• Verb ? serves ?x?y ?e ISA(e,Serving) ?
  Server(e,y) ? Served(e,x)
• VP ? Verb NP Verb.sem(NP.sem)
• S ? NP VP VP.sem(NP.sem)
• application of lambda expression lambda
  expression

26
Quasi-Logical Form

• During semantic analysis, we may use quantified
  expressions as terms. In this case, our formula
  will not be a FOPC formula. We call this form of
  formulas as quasi-logical form.
• A quasi-logical form should be converted into a
  normal FOPC formula by applying simple syntactic
  translations.
• Server(e,lt?x ISA(x,Restaurant)gt) a
  quasi-logical formula
• ?
• ?x ISA(x,Restaurant ) ? Server(e,x) a normal
  FOPC formula

27
Parse Tree with Logical Forms
28
Pros and Cons

• If you integrate semantic analysis into the
  parser as it is running
• You can use semantic constraints to cut off
  parses that make no sense
• But you assign meaning representations to
  constituents that dont take part in the correct
  (most probable) parse

29
Complex Terms

• Allow the compositional system to pass around
  representations like the following as objects
  with parts
• Complex-Term ? ltQuantifier var bodygt

30
Example

• Our restaurant example winds up looking like
• Big improvement

31
Conversion

• So complex terms wind up being embedded inside
  predicates. So pull them out and redistribute the
  parts in the right way
• P(ltquantifier, var, bodygt)
• turns into
• Quantifier var body connective P(var)

32
Example
33
Quantifiers and Connectives

• If the quantifier is an existential, then the
  connective is an (and)
• If the quantifier is a universal, then the
  connective is an
• -gt (implies)

34
Multiple Complex Terms

• Note that the conversion technique pulls the
  quantifiers out to the front of the logical form
• That leads to ambiguity if theres more than one
  complex term in a sentence.

35
Quantifier Ambiguity

• Consider
• Every restaurant has a menu
• That could mean that
• every restaurant has a menu
• Or that
• Theres some uber-menu out there and all
  restaurants have that menu

36
Quantifier Scope Ambiguity
37
Ambiguity

• This turns out to be a lot like the prepositional
  phrase attachment problem
• The number of possible interpretations goes up
  exponentially with the number of complex terms in
  the sentence
• The best we can do is to come up with weak
  methods to prefer one interpretation over another

38
Non-Compositionality

• Unfortunately, there are lots of examples where
  the meaning (loosely defined) cant be derived
  from the meanings of the parts
• Idioms, jokes, irony, sarcasm, metaphor,
  metonymy, indirect requests, etc

39
English Idioms

• Kick the bucket, buy the farm, bite the bullet,
  run the show, bury the hatchet, etc
• Lots of these constructions where the meaning of
  the whole is either
• Totally unrelated to the meanings of the parts
  (kick the bucket)
• Related in some opaque way (run the show)

40
The Tip of the Iceberg

• Describe this construction
• A fixed phrase with a particular meaning
• A syntactically and lexically flexible phrase
  with a particular meaning
• A syntactically and lexically flexible phrase
  with a partially compositional meaning

41
Example

• Enron is the tip of the iceberg.
• NP -gt the tip of the iceberg
• Not so good attested examples
• the tip of Mrs. Fords iceberg
• the tip of a 1000-page iceberg
• the merest tip of the iceberg
• How about
• Thats just the icebergs tip.

42
Example

• What we seem to need is something like
• NP -gt
• An initial NP with tip as its head followed by
• a subsequent PP with of as its head and that has
  iceberg as the head of its NP
• And that allows modifiers like merest, Mrs. Ford,
  and 1000-page to modify the relevant semantic
  forms

43
Quantified Phrases

• Consider
• A restaurant serves meat.
• Assume that A restaurant looks like
• If we do the normal lambda thing we get

44
Semantic analysis

• Goal to form the formal structures from smaller
  pieces
• Three approaches
• Syntax-driven semantic analysis
• Semantic grammar
• Information extraction filling templates

45
Semantic grammar

• Syntactic parse trees only contain parts that are
  unimportant in semantic processing.
• Ex Mary wants to go to eat some Italian food
• Rules in a semantic grammar
• InfoRequest ?USER want to go to eat FOODTYPE
• FOODTYPE?NATIONALITY FOODTYPE
• NATIONALITY?Italian/Mexican/.

46
Semantic grammar (cont)

• Pros
• No need for syntactic parsing
• Focus on relevant info
• Semantic grammar helps to disambiguate
• Cons
• The grammar is domain-specific.

47
Information extraction

• The desired knowledge can be described by a
  relatively simple and fixed template.
• Only a small part of the info in the text is
  relevant for filling the template.
• No full parsing is needed chunking, NE tagging,
  pattern matching,
• IE is a big field e.g., MUC. KnowItAll

48
Summary of semantic analysis

• Goal to form the formal structures from smaller
  pieces
• Three approaches
• Syntax-driven semantic analysis
• Semantic grammar
• Information extraction

49
Lexical Semantics
50
Meaning

• Traditionally, meaning in language has been
  studied from
• three perspectives
• The meaning of a text or discourse
• The meanings of individual sentences or
  utterances
• The meanings of individual words
• We started in the middle, now well look at the
  meanings of
• individual words.

51
Word Meaning

• We didnt assume much about the meaning of words
  when
• we talked about sentence meanings
• Verbs provided a template-like predicate argument
  structure
• Nouns were practically meaningless constants
• There has be more to it than that
• The internal structure of words that determines
• where they can go and what they can do
  (syntagmatic)

52
Whats a word?

• Words? Types, tokens, stems, roots, inflected
  forms?
• Lexeme
• An entry in a lexicon consisting of a pairing
  of a form with
• a single meaning representation
• Lexicon - A collection of lexemes

53
Lexical Semantics

• The linguistic study of systematic meaning
  related structure of lexemes is called Lexical
  Semantics.
• A lexeme is an individual entry in the lexicon.
• A lexicon is meaning structure holding meaning
  relations of lexemes.
• A lexeme may have different meanings. A lexemes
  meaning component is known as one of its senses.
• Different senses of the lexeme duck.
• an animal, to lower the head, ...
• Different senses of the lexeme yüz
• face, to swim, to skin, the front of something,
  hundred, ...

54
Relations Among Lexemes and Their Senses

• Homonymy
• Polysemy
• Snonymy
• Hyponymy
• Hypernym

55
Homonymy

• Homonymy is a relation that holds between words
  having the same form (pronunciation, spelling)
  with unrelated meanings.
• Bank -- financial institution, river bank
• Bat -- (wooden stick-like thing) vs (flying
  scary mammal thing)
• Fluke
• A fish, and a flatworm.
• The end parts of an anchor.
• The fins on a whale's tail.
• A stroke of luck.
• Homograph disambiguation is critically important
  in speech synthesis, natural language processing
  and other fields.

56
Polysemy

• Polysemy is the phenomenon of multiple related
  meanings in a same lexeme.
• Bank -- financial institution, blood bank, a
  synonym for 'rely upon'
• -- these senses are related.
• "I'm your friend, you can bank on me"
• While some banks furnish sperm only to married
  women, others are less restrictive
• However a river bank is a homonym to 1 and 2, as
  they do not share etymologies. It is a completely
  different meaning
• Mole - a small burrowing mammal
• several different entities called moles which
  refer to different things, but their names derive
  from 1.e.g. A Mole (espionage) burrows for
  information hoping to go undetected. .

57
Polysemy

• Milk
• The verb milk (e.g. "he's milking it for all he
  can get") derives from the process of obtaining
  milk.
• Lexicographers define polysemes within a single
  dictionary lemma, numbering different meanings,
  while homonyms are treated in separate lemmata.
• Most non-rare words have multiple meanings
• The number of meanings is related to its
  frequency
• Verbs tend more to polysemy
• Distinguishing polysemy from homonymy isnt
  always easy
• (or necessary)

58
Synonymy

• Synonymy is the phenomenon of two different
  lexemes having
• the same meaning.
• Big and large
• In fact, one of the senses of two lexemes are
  same.
• There arent any true synonyms.
• Two lexemes are synonyms if they can be
  successfully substituted
• for each other in all situations
• What does successfully mean?
• Preserves the meaning
• But may not preserve the acceptability based on
  notions of politeness, slang, ...
• Example - Big and large?
• Thats my big sister a big plane
• Thats my large sister a large plane

59
Hyponymy and Hypernym

• Hyponymy one lexeme denotes a subclass of the
  other lexeme.
• The more specific lexeme is a hyponymy of the
  more general lexeme.
• The more general lexeme is a hypernym of the more
  specific lexeme.
• A hyponymy relation can be asserted between two
  lexemes when the meanings of the lexemes entail a
  subset relation
• Since dogs are canids
• Dog is a hyponym of canid and
• Canid is a hypernym of dog
• Car is a hyponymy of vehicle, vehicle is a
  hypernym of car.

60
Ontology

• The term ontology refers to a set of distinct
  objects resulting from analysis of a domain.
• A taxonomy is a particular arrangements of the
  elements of an ontology into a
  tree-like class inclusion structure.
• A lexicon holds different senses of lexemes
  together with other relations among lexemes.

61
Lexical Resourses

• There are lots of lexical resources available
• Word lists
• On-line dictionaries
• Corpora
• The most ambitious one is WordNet
• A database of lexical relations for English
• Versions for other languages are under development

62
WordNet

• WordNet is widely used lexical database for
  English.
• WebPage http//www.cogsci.princeton.edu/wn/
• It holds
• The senses of the lexemes
• holds relations among nouns such as hypernym,
  hyponym, MemberOf, ..
• Holds relations among verbs such as hypernym,
• Relations are held for each different senses of a
  lexeme.

63
WordNet Relations

• Some of WordNet Relations (for nouns)

64
WordNet Hierarchies

• Hyponymy chains for the senses of the lexeme bass

65
WordNet - bass

• The noun "bass" has 8 senses in WordNet.1. bass
  -- (the lowest part of the musical range)2.
  bass, bass part -- (the lowest part in polyphonic
  music)3. bass, basso -- (an adult male singer
  with the lowest voice)4. sea bass, bass -- (the
  lean flesh of a saltwater fish of the family
  Serranidae)5. freshwater bass, bass -- (any of
  various North American freshwater fish with lean
  flesh (especially of the genus Micropterus))6.
  bass, bass voice, basso -- (the lowest adult male
  singing voice)7. bass -- (the member with the
  lowest range of a family of musical
  instruments)8. bass -- (nontechnical name for
  any of numerous edible marine and freshwater
  spiny-finned fishes)
• The adjective "bass" has 1 sense in WordNet.1.
  bass, deep -- (having or denoting a low vocal or
  instrumental range "a deep voice" "a bass voice
  is lower than a baritone voice" "a bass
  clarinet")

66
WordNet bass Hyponyms

• Results for "Hyponyms (...is a kind of this),
  full" search of noun "bass"6 of 8 senses of bass 
                                                    
  Sense 2bass, bass part -- (the lowest part in po
  lyphonic music)       gt ground bass -- (a short 
  melody in the bass that is constantly repeated)  
       gt thorough bass, basso continuo -- (a bass p
  art written out in full and accompanied by figures
   for successive chords)       gt figured bass -- 
  (a bass part in which the notes have numbers under
   them to indicate the chords to be played)Sense 4
  sea bass, bass -- (the lean flesh of a saltwater 
  fish of the family Serranidae)       gt striped b
  ass, striper -- (caught along the Atlantic coast o
  f the United States)Sense 5freshwater bass, bass
   -- (any of various North American freshwater fish
   with lean flesh (especially of the genus Micropte
  rus))       gt largemouth bass -- (flesh of large
  mouth bass)       gt smallmouth bass -- (flesh of
   smallmouth bass)Sense 6bass, bass voice, basso 
  -- (the lowest adult male singing voice)       gt
   basso profundo -- (a very deep bass voice)Sense 
  7bass -- (the member with the lowest range of a f
  amily of musical instruments)       gt bass fiddl
  e, bass viol, bull fiddle, double bass, contrabass
  , string bass -- (largest and lowest member of the
   violin family)       gt bass guitar -- (the lowe
  st six-stringed guitar)       gt bass horn, sousa
  phone, tuba -- (the lowest brass wind instrument)
             gt euphonium -- (a bass horn (brass win
  d instrument) that is the tenor of the tuba family
  )           gt helicon, bombardon -- (a tuba that
   coils over the shoulder of the musician)       
  gt bombardon, bombard -- (a large shawm the bass m
  ember of the shawm family)Sense 8bass -- (nontec
  hnical name for any of numerous edible marine and 
  freshwater spiny-finned fishes)       gt freshwat
  er bass -- (North American food and game fish)

67
WordNet bass Synonyms

• Results for "Synonyms, ordered by estimated
  frequency" search of noun "bass"8 senses of bass 
                                                    
       Sense 1bass -- (the lowest part of the musi
  cal range)       gt low pitch, low frequency -- (
  a pitch that is perceived as below other pitches)
  Sense 2bass, bass part -- (the lowest part in pol
  yphonic music)       gt part, voice -- (the melod
  y carried by a particular voice or instrument in p
  olyphonic music "he tried to sing the tenor part"
  )Sense 3bass, basso -- (an adult male singer wit
  h the lowest voice)       gt singer, vocalist, vo
  calizer, vocaliser -- (a person who sings)Sense 4
  sea bass, bass -- (the lean flesh of a saltwater 
  fish of the family Serranidae)       gt saltwater
   fish -- (flesh of fish from the sea used as food)
  Sense 5freshwater bass, bass -- (any of various 
  North American freshwater fish with lean flesh (es
  pecially of the genus Micropterus))       gt fres
  hwater fish -- (flesh of fish from fresh water use
  d as food)Sense 6bass, bass voice, basso -- (the
   lowest adult male singing voice)       gt singin
  g voice -- (the musical quality of the voice while
   singing)Sense 7bass -- (the member with the low
  est range of a family of musical instruments)    
     gt musical instrument, instrument -- (any of va
  rious devices or contrivances that can be used to 
  produce musical tones or sounds)Sense 8bass -- (
  nontechnical name for any of numerous edible marin
  e and freshwater spiny-finned fishes)       gt pe
  rcoid fish, percoid, percoidean -- (any of numerou
  s spiny-finned fishes of the order Perciformes)

68
Internal Structure of Words

• Paradigmatic relations connect lexemes together
  in particular ways
• but dont say anything about what the meaning
  representation of
• a particular lexeme should consist of.
• Various approaches have been followed to describe
  
• the semantics of lexemes.
• Thematic roles in predicate-bearing lexemes
• Selection restrictions on thematic roles
• Decompositional semantics of predicates
• Feature-structures for nouns

69
Thematic Roles

• Thematic roles provide a shallow semantic
  language for characterizing certain arguments of
  verbs.
• For example
• Ali broke the glass.
• Veli opened the door.
• Ali is Breaker and the glass is BrokenThing of
  Breaking event
• Veli is Opener and the door is OpenedThing of
  Opening event.
• These are deep roles of arguments of events.
• Both of these events have actors which are doer
  of a volitional event, and things affected by
  this action.
• A thematic role is a way of expressing this kind
  of commonality.
• AGENT and THEME are thematic roles.

70
Some Thematic Roles

• AGENT --The volitional causer of an event -- She
  broke the door
• EXPERIENCER -- The experiencer of an event -- Ali
  has a headache.
• FORCE -- The non-volitional causer of an event --
  The wind blows it.
• THEME -- The participant most directly effected
  by an event --
• She broke the door.
• INSTRUMENT -- An instrument used in an event --
• He opened it with a knife.
• BENEFICIARY -- A beneficiary of an event -- I
  bought it for her.
• SOURCE -- The origin of the object of a transfer
  event --
• I flew from Rome.
• GOAL -- The destination of the object of a
  transfer event --
• I flew to Ankara.

71
Thematic Roles (cont.)

• Takes some of the work away from the verbs.
• Its not the case that every verb is unique and
  has to completely specify how all of its
  arguments uniquely behave.
• Provides a mechanism to organize semantic
  processing
• It permits us to distinguish near surface-level
  semantics
• from deeper semantics

72
Linking

• Thematic roles, syntactic categories and their
  positions in larger syntactic structures are all
  intertwined in complicated ways.
• For example
• AGENTS are often subjects
• In a VP-gtV NP NP rule, the first NP is often a
  GOAL
• and the second a THEME

73
Deeper Semantics

• He melted her reserve with a husky-voiced paean
  to her eyes.
• If we label the constituents He and her reserve
  as the Melter and Melted, then those labels lose
  any meaning they might have had.
• If we make them Agent and Theme then we dont
  have the same problems

74
Selectional Restrictions

• A selectional restriction augments thematic roles
  by allowing lexemes to place certain semantic
  restrictions on the lexemes and phrases can
  accompany them in a sentence.
• I want to eat someplace near Bilkent.
• Now we can say that eat is a predicate that has
  an AGENT and a THEME
• And that the AGENT must be capable of eating and
  the THEME must be capable of
• being eaten
• Each sense of a verb can be associated with
  selectional restrictions.
• THY serves NewYork. -- direct object (theme) is
  a place
• THY serves breakfast. -- direct object (theme) is
  a meal.
• We may use these selectional restrictions to
  disambiguate a sentence.

75
As Logical Statements

• For eat
• Eating(e) Agent(e,x) Theme(e,y)Isa(y, Food)
• (adding in all the right quantifiers and lambdas)

76
WordNet

• Use WordNet hyponyms (type) to encode the
  selection restrictions

77
Specificity of Restrictions

• What can you say about THEME in each with respect
  to the verb?
• Some will be high up in the WordNet hierarchy,
  others not so high
• PROBLEMS
• Unfortunately, verbs are polysemous and language
  is creative
• ate glass on an empty stomach accompanied only
  by water
• and tea
• you cant eat gold for lunch if youre hungry
• get it to try to eat Afghanistan

78
Discovering the Restrictions

• Instead of hand-coding the restrictions for each
  verb,
• can we discover a verbs restrictions by using a
  corpus and WordNet?
• Parse sentences and find heads
• Label the thematic roles
• Collect statistics on the co-occurrence of
  particular headwords with particular thematic
  roles
• Use the WordNet hypernym structure to find the
  most meaningful level to use as a restriction

79
Motivation

• Find the lowest (most specific) common ancestor
  that covers a significant number of the examples

80
Word-Sense Disambiguation

• Word sense disambiguation refers to the process
  of selecting
• the right sense for a word from among the senses
  that the word is known to have
• Semantic selection restrictions can be used to
  disambiguate
• Ambiguous arguments to unambiguous predicates
• Ambiguous predicates with unambiguous arguments
• Ambiguity all around

81
Word-Sense Disambiguation

• We can use selectional restrictions for
  disambiguation.
• He cooked simple dishes.
• He broke the dishes.
• But sometimes, selectional restrictions will not
  be enough to disambiguate.
• What kind of dishes do you recommend? -- we
  cannot know what sense is used.
• There can be two lexemes (or more) with multiple
  senses.
• They serve vegetarian dishes.
• Selectional restrictions may block the finding of
  meaning.
• If you want to kill Turkey, eat its banks.
• Kafayi yedim.
• These situations leave the system with no
  possible meanings, and they can indicate a
  metaphor.

82
WSD and Selection Restrictions

• Ambiguous arguments
• Prepare a dish
• Wash a dish
• Ambiguous predicates
• Serve Denver
• Serve breakfast
• Both
• Serves vegetarian dishes

83
WSD and Selection Restrictions

• This approach is complementary to the
  compositional analysis approach.
• You need a parse tree and some form of
  predicate-argument analysis derived from
• The tree and its attachments
• All the word senses coming up from the lexemes at
  the leaves of the tree
• Ill-formed analyses are eliminated by noting any
  selection restriction violations

84
Problems

• As we saw last time, selection restrictions are
  violated all the time.
• This doesnt mean that the sentences are
  ill-formed or preferred less than others.
• This approach needs some way of categorizing and
  dealing with the various ways that restrictions
  can be violated

85
WSD Tags

• Whats a tag?
• A dictionary sense?
• For example, for WordNet an instance of bass in
  a text has 8 possible tags or labels (bass1
  through bass8).

86
WordNet Bass

• The noun bass'' has 8 senses in WordNet
• bass - (the lowest part of the musical range)
• bass, bass part - (the lowest part in polyphonic
  music)
• bass, basso - (an adult male singer with the
  lowest voice)
• sea bass, bass - (flesh of lean-fleshed saltwater
  fish of the family Serranidae)
• freshwater bass, bass - (any of various North
  American lean-fleshed freshwater fishes
  especially of the genus Micropterus)
• bass, bass voice, basso - (the lowest adult male
  singing voice)
• bass - (the member with the lowest range of a
  family of musical instruments)
• bass -(nontechnical name for any of numerous
  edible marine and
• freshwater spiny-finned fishes)

87
Representations

• Most supervised ML approaches require a very
  simple representation for the input training
  data.
• Vectors of sets of feature/value pairs
• I.e. files of comma-separated values
• So our first task is to extract training data
  from a corpus with respect to a particular
  instance of a target word
• This typically consists of a characterization of
  the window of text surrounding the target

88
Representations

• This is where ML and NLP intersect
• If you stick to trivial surface features that are
  easy to extract from a text, then most of the
  work is in the ML system
• If you decide to use features that require more
  analysis (say parse trees) then the ML part may
  be doing less work (relatively) if these features
  are truly informative

89
Surface Representations

• Collocational and co-occurrence information
• Collocational
• Encode features about the words that appear in
  specific positions to the right and left of the
  target word
• Often limited to the words themselves as well as
  theyre part of speech
• Co-occurrence
• Features characterizing the words that occur
  anywhere in the window regardless of position
• Typically limited to frequency counts

90
Collocational

• Position-specific information about the words in
  the window
• guitar and bass player stand
• guitar, NN, and, CJC, player, NN, stand, VVB
• In other words, a vector consisting of
• position n word, position n part-of-speech

91
Co-occurrence

• Information about the words that occur within the
  window.
• First derive a set of terms to place in the
  vector.
• Then note how often each of those terms occurs in
  a given window.

92
Classifiers

• Once we cast the WSD problem as a classification
  problem, then all sorts of techniques are
  possible
• Naïve Bayes (the right thing to try first)
• Decision lists
• Decision trees
• Neural nets
• Support vector machines
• Nearest neighbor methods

93
Classifiers

• The choice of technique, in part, depends on the
  set of features that have been used
• Some techniques work better/worse with features
  with numerical values
• Some techniques work better/worse with features
  that have large numbers of possible values
• For example, the feature the word to the left has
  a fairly large number of possible values

94
Statistical Word-Sense Disambiguation
Where s is a vector of senses, V is the vector
representation of the input
By Bayesian rule
By making independence assumption of meanings.
This means that the result is the product of the
probabilities of its individual features given
that its sense
95
Problems

• Given these general ML approaches, how many
  classifiers do I need to perform WSD robustly
• One for each ambiguous word in the language
• How do you decide what set of tags/labels/senses
  to use for a given word?
• Depends on the application

96
END
97
Examples from RussellNorvig (1)

• 7.2. p.213
• Not all students take both History and Biology.
• Only one student failed History.
• Only one student failed both History and Biology.
• The best history in History was better than the
  best score in Biology.
• Every person who dislikes all vegetarians is
  smart.
• No person likes a smart vegetarian.
• There is a woman who likes all men who are
  vegetarian.
• There is a barber who shaves all men in town who
  don't shave themselves.
• No person likes a professor unless a professor is
  smart.
• Politicians can fool some people all of the time
  or all people some of the time but they cannot
  fool all people all of the time.

98
Categories Events

• Categories
• VegetarianRestaurant (Joes) categories are
  relations and not objects
• MostPopular(Joes,VegetarianRestaurant) not
  FOPC!
• ISA (Joes,VegetarianRestaurant) reification
  (turn all concepts into objects)
• AKO (VegetarianRestaurant,Restaurant)
• Events
• Reservation (Hearer,Joes,Today,8PM,2)
• Problems
• Determining the correct number of roles
• Representing facts about the roles associated
  with an event
• Ensuring that all the correct inferences can be
  drawn
• Ensuring that no incorrect inferences can be drawn

99
MUC-4 Example
100
Subcategorization frames

• I ate
• I ate a turkey sandwich
• I ate a turkey sandwich at my desk
• I ate at my desk
• I ate lunch
• I ate a turkey sandwich for lunch
• I ate a turkey sandwich for lunch at my desk
• - no fixed arity (problem for FOPC)

101
One possible solution

• Eating1 (Speaker)
• Eating2 (Speaker, TurkeySandwich)
• Eating3 (Speaker, TurkeySandwich, Desk)
• Eating4 (Speaker, Desk)
• Eating5 (Speaker, Lunch)
• Eating6 (Speaker, TurkeySandwich, Lunch)
• Eating7 (Speaker, TurkeySandwich, Lunch, Desk)
• Meaning postulates are used to tie semantics of
  predicates ? w,x,y,z Eating7(w,x,y,z) ?
  Eating6(w,x,y)
• Scalability issues again!

102
Another solution

• - Say that everything is a special case of
  Eating7 with some arguments unspecified
• ?w,x,y Eating (Speaker,w,x,y)
• - Two problems again
• Too many commitments (e.g., no eating except at
  meals lunch, dinner, etc.)
• No way to individuate events ?w,x Eating
  (Speaker,w,x,Desk) ?w,y Eating
  (Speaker,w,Lunch,y) cannot combine into ?w
  Eating (Speaker,w,Lunch,Desk)

103
Reification

• ? w Isa(w,Eating) ? Eater(w,Speaker) ?
  Eaten(w,TurkeySandwich) equivalent to sentence
  5.
• Reification
• No need to specify fixed number of arguments for
  a given surface predicate
• No more roles are postulated than mentioned in
  the input
• No need for meaning postulates to specify logical
  connections among closely related examples

104
Representing time

• I arrived in New York
• I am arriving in New York
• I will arrive in New York
• ? w Isa(w,Arriving) ? Arriver(w,Speaker) ?
  Destination(w,NewYork)

105
Representing time

• ? i,e,w,t Isa(w,Arriving) ? Arriver(w,Speaker) ?
  Destination(w,NewYork) ? IntervalOf(w,i) ?
  EndPoint(I,e) ? Precedes (e,Now)
• ? i,e,w,t Isa(w,Arriving) ? Arriver(w,Speaker) ?
  Destination(w,NewYork) ? IntervalOf(w,i) ?
  MemberOf(i,Now)
• ? i,e,w,t Isa(w,Arriving) ? Arriver(w,Speaker) ?
  Destination(w,NewYork) ? IntervalOf(w,i) ?
  StartPoint(i,s) ? Precedes (Now,s)

106
Representing time

• We fly from San Francisco to Boston at 10.
• Flight 1390 will be at the gate an hour now.
• Use of tenses
• Flight 1902 arrived late.
• Flight 1902 had arrived late.
• similar tenses
• When Marys flight departed, I ate lunch
• When Marys flight departed, I had eaten lunch
• reference point

107
Aspect

• Stative I know my departure gate
• Activity John is flyingno particular end point
• Accomplishment Sally booked her flightnatural
  end point and result in a particular state
• Achievement She found her gate
• Figuring out statives I am needing the
  cheapest fare. I am wanting to go today. Need
  the cheapest fare!

108
Representing beliefs

• Want, believe, imagine, know - all introduce
  hypothetical worlds
• I believe that Mary ate British food.
• Reified example
• ? u,v Isa(u,Believing) ? Isa(v,Eating) ?
  Believer (u,Speaker) ? BelievedProp(u,v) ?
  Eater(v,Mary) ? Eaten(v,BritishFood)
• However this implies also
• ? u,v Isa(v,Eating) ? Eater(v,Mary) ?
  Eaten(v,BritishFood)
• Modal operators
• Believing(Speaker,Eating(Mary,BritishFood)) -
  not FOPC! predicates in FOPC hold between
  objects, not between relations.
• Believes(Speaker, ? v ISA(v,Eating) ?
  Eater(v,Mary) ? Eaten(v,BritishFood))

109
Modal operators

• Beliefs
• Knowledge
• Assertions
• Issues If you are interested in baseball, the
  Red Sox are playing tonight.

110
Examples from RussellNorvig (2)

• 7.3. p.214
• One more outburst like that and you'll be in
  comptempt of court.
• Annie Hall is on TV tonight if you are
  interested.
• Either the Red Sox win or I am out ten dollars.
• The special this morning is ham and eggs.
• Maybe I will come to the party and maybe I won't.
• Well, I like Sandy and I don't like Sandy.