Subjectivity and Sentiment Analysis

1
Subjectivity and Sentiment Analysis

• Jan Wiebe
• Department of Computer Science
• Intelligent Systems Program
• University of Pittsburgh

2
Main Collaborators in the Work Described Today

• Claire Cardie, Cornell University
• Rada Mihalcea, University of North Texas
• Ellen Riloff, University of Utah
• Swapna Somasundaran, U. Pittsburgh
• Theresa Wilson, University of Edinburgh

3
Burgeoning Field

• Quite a large problem space
• Several terms reflecting varying goals and models
• Sentiment Analysis
• Opinion Mining
• Opinion Extraction
• Subjectivity Analysis
• Appraisal Analysis
• Affect Sensing
• Emotion Detection
• Identifying Perspective
• Etc.

4
What is Subjectivity?

• The linguistic expression of somebodys emotions,
  evaluations, beliefs, speculations, intentions,
  etc.
• Here, sentiment is a type of subjectivity
  (positive and negative emotions and evaluations)

5
What is Subjectivity?

• The linguistic expression of somebodys emotions,
  evaluations, beliefs, speculations, intentions,
  etc.

6
Subjectivity and Sentiment Analysis

• Automatic extraction of peoples sentiments,
  opinions, etc. expressed in text (newspapers,
  blogs, etc)

7
Applications

• Product review mining Based on what people write
  in their reviews, what features of the ThinkPad
  T43 do they like and which do they dislike?
• Review classification Is a review positive or
  negative toward the movie?
• Tracking sentiments toward topics over time
  Based on sentiments expressed in text, is anger
  ratcheting up or cooling down?
• Prediction (election outcomes, market trends)
  Based on opinions expressed in text, will Clinton
  or Obama win?
• Etcetera!

8
Focus

• Subjective language is highly ambiguous
• Simple keyword approaches are severely limited
• Our focus is linguistic disambiguation how
  should language be interpreted?
• Is it subjective in the first place? If so, is
  it positive or negative? How intense is it? Etc.

9
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
10
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
11
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
The dream
12
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
The dream
NLP methods/resources building toward
full interpretations
13
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
The dream
NLP methods/resources building toward
full interpretations
Today 4 problems in subjectivity analysis along
the continuum
14
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
15
Dictionary Definitions senses

• Interest, involvement -- (a sense of concern
  with and curiosity about someone or something
  "an interest in music")
• Interest -- (a fixed charge for borrowing
  money usually a percentage of the amount
  borrowed "how much interest do you pay on your
  mortgage?")

16
Dictionary Definitions senses

• Interest, involvement -- (a sense of concern
  with and curiosity about someone or something
  "an interest in music")
• Interest -- (a fixed charge for borrowing
  money usually a percentage of the amount
  borrowed "how much interest do you pay on your
  mortgage?")

17
Senses

• Most approaches to subjectivity and sentiment
  analysis exploit subjectivity lexicons, which are
  lists of keywords that have been gathered
  together because they have subjective usages
• Even in subjectivity lexicons, many senses of the
  keywords are objective -- 50 in our study!
• So, many appearances of keywords in texts are
  false hits

18
Senses

• His alarm grew as the election returns came in.
• He forgot to set his alarm.
• His trust grew as the candidate spoke.
• His trust grew as interest rates increased.

19
Subjectivity Sense Labeling

• Automatically classifying senses as subjective or
  objective, and classifying subjective senses by
  polarity

20
WSD using Subjectivity Tagging
21
WSD using Subjectivity Tagging
He spins a riveting plot which grabs and holds
the readers interest.
S
Sense 4 Sense 1?
Sense 4 a sense of concern with and curiosity
about someone or something S Sense 1 a fixed
charge for borrowing money O
Subjectivity Classifier
WSD System
Sense 1 Sense 4?
O
The notes do not pay interest.
22
WSD using Subjectivity Tagging
He spins a riveting plot which grabs and holds
the readers interest.
S
Sense 4 Sense 1?
Sense 4 a sense of concern with and curiosity
about someone or something S Sense 1 a fixed
charge for borrowing money O
Subjectivity Classifier
WSD System
Sense 1 Sense 4?
O
The notes do not pay interest.
23
Subjectivity Tagging using WSD
Subjectivity Classifier
He spins a riveting plot which grabs and holds
the readers interest.
S O?
O S?
The notes do not pay interest.
24
Subjectivity Tagging using WSD
Subjectivity Classifier
He spins a riveting plot which grabs and holds
the readers interest.
S O?
Sense 4
WSD System
O S?
Sense 1
The notes do not pay interest.
25
Subjectivity Tagging using WSD
Subjectivity Classifier
He spins a riveting plot which grabs and holds
the readers interest.
S O?
Sense 4
WSD System
O S?
Sense 1
The notes do not pay interest
26
Methods for Subjectivity Sense Labeling

• Corpus based
• Lexicon based (exploiting knowledge in WordNet)
• Integration

27
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
28
Learning Subjective Language from Corpora

• There is a seemingly endless variety of
  subjective expressions, i.e., expressions that
  may be used to express opinions and sentiments
• Many do not correspond to dictionary definitions
• Subjective language varies among different types
  of corpora

29
Learning Subjective Language from Corpora

• Goal create subjective language learners that
  do not require manually annotated texts as input
• Learners may be applied to
• large text collections to generate more expansive
  dictionaries
• domain specific corpora with specialized
  vocabularies
• Methods weakly supervised information extraction
  methods

30
Information Extraction

• Information extraction (IE) systems identify
  facts related to a domain of interest.
• Extraction patterns are lexico-syntactic
  expressions that identify the role of an object.
  For example

ltsubjectgt was killed
assassinated ltdobjgt
murder of ltnpgt
31
Learning Subjective Language

• Use IE techniques to learn subjective nouns
• Use IE techniques to learn subjective patterns

32
Learning Subjective Nouns

• Hypothesis extraction patterns can identify
  subjective contexts that co-occur with subjective
  nouns

Example expressed ltdobjgt
concern, hope, support
33
Learning Subjective Nouns

• Method IE-based bootstrapping algorithms
  designed to learn semantic categories

34
Extraction Examples
expressed ltdobjgt condolences, hope, grief,
views, worries indicative of ltnpgt compromise,
desire, thinking inject ltdobjgt vitality,
hatred reaffirmed ltdobjgt resolve, position,
commitment voiced ltdobjgt outrage, support,
skepticism, opposition, gratitude,
indignation show of ltnpgt support, strength,
goodwill, solidarity ltsubjgt was shared anxiety,
view, niceties, feeling
35
Meta-Bootstrapping Riloff Jones 99
Ex hope, grief, joy, concern, worries
Ex expressed ltDOBJgt
Best Extraction Pattern
Ex happiness, relief, condolences
Extractions (Nouns)
36
Subjective Seed Words
37
Examples of Learned Nouns
anguish exploitation pariah antagonism
evil repudiation apologist fallacies
revenge atrocities genius
rogue barbarian goodwill
sanctimonious belligerence
humiliation scum bully ill-treatment
smokescreen condemnation injustice
sympathy denunciation innuendo
tyranny devil insinuation
venom diatribe liar exaggeration
mockery
38
Learning Subjective Patterns

• Extraction patterns can represent linguistic
  expressions that are not fixed word sequences.

drove NP up the wall - drove him up the
wall - drove George Bush up the wall - drove
George Herbert Walker Bush up the wall
step on modifiers toes - step on her toes -
step on the mayors toes - step on the newly
elected mayors toes
39
Learning Subjective Patterns

• Method IE-based techniques for learning
  extraction patterns

40
Learning Subjective Patterns

• Method IE-based techniques for learning
  extraction patterns themselves

41
Patterns with Interesting Behavior
PATTERN FREQ P(Subj Pattern) ltsubjgt asked 128
.63 ltsubjgt was asked 11 1.0
42
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
43
Recognizing Contextual Polarity in Phrase-Level
Sentiment Analysis
44
Prior versus Contextual Polarity

• Several subjectivity lexicons include polarity
  information
• beautiful ? positive
• horrid ? negative
• In context, words often appear in phrases with
  the opposite polarity

Cheers to Timothy Whitfield for the wonderfully
horrid visuals.
45
Recognizing Contextual Polarity

• Goal given a phrase containing a word from the
  lexicon, is it subjective? If so, is it positive
  or negative?
• Method machine learning with a variety of
  features

46
Features

• Binary features
• In subject
• The human rights report
• poses
• In copular
• I am confident
• In passive voice
• must be regarded

Etcetera
47
Contextual Polarity is Complex

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

48
Contextual Polarity is Complex

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

49
Contextual Polarity is Complex

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

50
Contextual Polarity is Complex

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

51
Features

• General polarity shifter
• have few risks/rewards
• Negative polarity shifter
• lack of understanding
• Positive polarity shifter
• abate the damage

• Contextual valence shifters

52
Features

• Structural features

• Binary features
• In subject
• The human rights report
• poses
• In copular
• I am confident
• In passive voice
• must be regarded

53
Features

• Modification features

• Modifies polarity
• substantial negative
• Modified by polarity
• challenge positive

54
Features

• Negation features

• Binary features
• Negated
• For example
• not good
• does not look very good
• not only good but amazing
• Negated subject
• No politically prudent Israeli could support
  either of them.

55
Features

• Etc.

56
Interpretation
Dictionary definition meanings purely out of
context
Full contextual Interpretation of words in text
continuum
57
Discourse-Level Opinion Interpretation

• Interpretations involving multiple sentences
  within the discourse
• Opinion Frames are composed of 2 opinions and the
  relation between their targets (what they are
  opinions of)
• Larger structures emerge from interdependent
  frames

58
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD

Sentiment opinions include positive and negative
evaluations, emotions, and judgments
Arguing opinion include arguing for or against
something, and arguing that something should or
should not be done
59
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD

targets what the opinion is about
60
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD
• I think the LCD is hot

61
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD
• I think the LCD is hot

Joint Interpretation of opinions in the discourse
62
Discourse-Level Opinion Interpretation

• Goal recognize opinion frames
• Method develop individual classifiers for their
  components, and then perform joint inference to
  improve performance

63
Opinion Frames Interdependent Interpretation
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
SP
S?
64
Opinion Frames Interdependent Interpretation
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
same
same
SP
S?
65
Opinion Frames Interdependent Interpretation
reinforcing
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
same
reinforcing
same
SP
S?
66
Opinion Frames Interdependent Interpretation
reinforcing
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
same
reinforcing
same
SP
S?
SPSPsame, SNSNsame, APAPsame, ANANsame, SPAPsame,
APSPsame, SNANsame, ANSNsame, SPSNalt, SNSPalt,
APANalt, ANAPalt, SPANalt, SNAPalt, APSNalt,
ANSPalt SPSNsame, SNSPsame, APANsame,
ANAPsame, SPANsame, APSNsame, SNAPsame,
ANSPsame, SPSPalt, SNSNalt, APAPalt,
ANANalt, SPAPalt, SNANalt, APSPalt, ANSNalt
reinforcing
non-reinforcing
67
Opinion Frames Interdependent Interpretation
reinforcing
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
same
reinforcing
same
SP
S?
SPSPsame, SNSNsame, APAPsame, ANANsame, SPAPsame,
APSPsame, SNANsame, ANSNsame, SPSNalt, SNSPalt,
APANalt, ANAPalt, SPANalt, SNAPalt, APSNalt,
ANSPalt SPSNsame, SNSPsame, APANsame,
ANAPsame, SPANsame, APSNsame, SNAPsame,
ANSPsame, SPSPalt, SNSNalt, APAPalt,
ANANalt, SPAPalt, SNANalt, APSPalt, ANSNalt
reinforcing
non-reinforcing
68
Opinion Frames Interdependent Interpretation
reinforcing
SP
D... this kind of rubbery material, its a bit
more bouncy, like you said they get chucked
around a lot. A bit more durable and that can
also be ergonomic and it kind of feels a bit
different from all the other remote controls.
SP
same
reinforcing
same
SP
SP
SPSPsame, SNSNsame, APAPsame, ANANsame, SPAPsame,
APSPsame, SNANsame, ANSNsame, SPSNalt, SNSPalt,
APANalt, ANAPalt, SPANalt, SNAPalt, APSNalt,
ANSPalt SPSNsame, SNSPsame, APANsame,
ANAPsame, SPANsame, APSNsame, SNAPsame,
Etc, SPSPalt, SNSNalt, APAPalt, ANANalt, SPAPalt,
SNANalt, APSPalt, ANSNalt
reinforcing
non-reinforcing
69
Manual Annotations

• I think people are happy because Chavez has
  fallen.

direct subjective span are happy source
ltwriter, I, Peoplegt attitude
direct subjective span think source
ltwriter, Igt attitude
inferred attitude span are happy because
Chavez has fallen type neg sentiment
intensity medium target
attitude span are happy type pos sentiment
intensity medium target
attitude span think type positive arguing
intensity medium target
target span people are happy because
Chavez has fallen
target span Chavez has fallen
target span Chavez
70
ltppolneggtcondemnlt/ppolgt ltppolposgtgreatlt/ppol
gt ltppolneggtwickedlt/ppolgt
Recognizing Context Polarity EMNLP05
ltgt lt/gtltgt lt/gt
ltgt lt/gtltgt lt/gt
ltgt lt/gtltgt lt/gt
ltgt lt/gtltgt lt/gt
ltcpolposgtwicked lt/cpolgt visuals
ltcpolneggtloudly condemnedlt/cpolgt
QA IE Opinion Tracking
The building has been ltsubjectivityobjgt
condemned lt/subjectivitygt
71
Other Recent Projects

• Learning Multilingual Subjective Language via
  Cross-Lingual Projections
• Universal representation of subjectivity clues
• Single words
• N-grams
• Word senses
• Lexico-syntactic patterns
• Broken into definitional and (standoff)
  attributional components
• Exploiting subjectivity analysis to improve
  Information extraction and automatic question
  answering systems

72
Pointers

• Please see http//www.cs.pitt.edu/wiebe
• Publications
• OpinionFinder
• Subjectivity lexicon
• MPQA manually annotated corpus
• Tutorials
• Bibliography

73
(General) Subjectivity TypesWilson 2008
Other (including cognitive) Note similar
ideas polarity, semantic orientation, sentiment
74
Acknowledgements

• CERATOPS Center for the Extraction and
  Summarization of Events and Opinions in Text
• Pittsburgh Paul Hoffmann, Josef Ruppenhofer,
  Swapna Somasundaran, Theresa Wilson
• Cornell Claire Cardie, Eric Breck, Yejin Choi,
  Ves Stoyanov
• Utah Ellen Riloff, Sidd Patwardhan, Bill
  Phillips
• UNT Rada Mihalcea, Carmen Banea
• NLP_at_Pitt Wendy Chapman, Rebecca Hwa, Diane
  Litman,