High Frequency Word Entrainment in Spoken Dialogue

1
High Frequency Word Entrainment in Spoken Dialogue
ACL, June 2008 - Columbus, OH

• Ani Nenkova - Agustín Gravano - Julia Hirschberg

2
Entrainment

• In conversation, people adapt the way they speak
  to match their partners.
• Entrainment, accommodation, adaptation,
  alignment, convergence.

3
Previous WorkExistence of entrainment

• In conversation, speakers
• Negotiate common ways of describing things.
• S.E. Brennan, 1996
• Alter their intensity to match their partners.
• R. Coulston et al., 2002
• A. Ward D. Litman, 2007
• Reuse syntactic constructions.
• D. Reitter et al., 2006

4
Previous WorkRole of entrainment

• Entrainment at different levels (lex, syn, sem)
• Is key for both production and understanding, and
  facilitates interaction.
• M.J. Pickering S. Garrod, 2004
• D. Goleman, 2006
• Is a good predictor of task success (MapTask).
• D. Reitter J. Moore, 2007

5
This Work

• Novel measures of entrainment based on usage of
  high-frequency words (HFW).
• Entrainment and
• Perceived naturalness
• Task success
• Dialogue coordination
• Implications in the development of Spoken
  Dialogue Systems.

6
High-Frequency Words

• Most common words in a corpus, or in a
  conversation.
• Typically, function words and cue words.
• Entrainment of HFW
• Domain-independent

7
Entrainment Naturalness

• Will a conversation be perceived as more natural
  if HFW entrainment occurs?
• Switchboard corpus
• 2430 spontaneous telephone conversations in
  American English
• Speakers asked to discuss a pre-assigned topic
• Annotated for degree of perceived naturalness,
  from 1 (Very natural) to 5 (Not natural at
  all).

8
Entrainment NaturalnessMeasure of Entrainment
Where fraction(w, Si) ? Fraction of times
Speaker i used word w in the
conversation Examples entr(okay) ? ? 10
/ 500 8 / 600 ? ? 0.0067 entr(yeah)
? ? 1 / 500 30 / 600 ? ? 0.048
9
Entrainment NaturalnessMachine Learning Task

• Predict the perceived naturalness of
  conversations.
• Binary decision, over balanced data
• 250 conversations rated 1 (very natural), and
  250 with ratings 3, 4 or 5.
• Computed entr(w) for the 100 most frequent words
  in the entire Switchboard corpus.
• Feature selection 25 most predictive words.
• um, how, okay, go, Ive, all, very, as, or, up,
  a, no, more, something, from, this, what, too,
  got, can, he, in, things, you, and.

10
Entrainment NaturalnessResults

• Logistic regression model (10-fold CV) 63.76
  accuracy (significantly better than 50 baseline)
• Entrainment in usage of HFW is a good indicator
  of perceived naturalness.

11
Entrainment Task Success

• Is a conversation more likely to succeed when HFW
  entrainment occurs?
• Columbia Games Corpus
• 12 spontaneous task-oriented dialogues in
  American English, with no eye contact.
• Each pair of subjects played a series of
  computer-based matching games.
• Subjects received a score after each task.

12
Entrainment Task SuccessMeasures of Entrainment
Where c Class of words
countSi(w) No. of times Si used word w in the
conversation
13
Entrainment Task SuccessWord Classes

• 25MF-G 25 most frequent words in the game
• 25MF-C 25 most frequent words in the corpus
• the, a, okay, and, of, I, on, right, is, it,
  that, have,
• ACW Affirmative cue words
• alright, mm-hm, okay, right, uh-huh, yeah, yes
• 7.9 of all words in the Games Corpus

14
Entrainment Task SuccessResults

• Correlations with game score
• HFW entrainment positively correlated with task
  success.

15
Entrainment Coordination

• Is dialogue more coordinated when HFW entrainment
  occurs?
• Columbia Games Corpus
• Labeled for type of turn exchanges (Beattie,
  1982), including
• Smooth Switch S2 starts his turn after S1 has
  finished hers
• Interruption S2 starts his turn before S1 has
  finished hers
• Overlap S2 starts his turn just before S1 has
  finished hers, but without interrupting.

16
Entrainment CoordinationResults

• Significant correlations (plt0.05)
• ENTR1(ACW) Prop. of Overlaps (cor 0.64)
• ENTR2(ACW) Prop. of Overlaps (cor 0.61)
• ENTR2(25MF-G) Prop. of Overlaps (cor
  0.60)
• ENTR1(25MF-C) Prop. of Interruptions (cor
  0.61)
• ENTR2(ACW) Mean Latency of Smooth
  Switches (cor 0.76)
• HFW entrainment positively correlated with more
  overlaps, fewer interruptions, and shorter
  inter-turn latencies.

17
Conclusion

• Two novel measures of lexical entrainment, based
  on the usage of high-frequency words.
• Entrainment in usage of high-frequency words is
  correlated with
• Perceived naturalness
• Task success
• Dialogue coordination
• Implications in the development of SDS.

18
High Frequency Word Entrainment in Spoken Dialogue
ACL, June 2008 - Columbus, OH

• Ani Nenkova - Agustín Gravano - Julia Hirschberg