Conversational role assignment problem in multi-party dialogues - PowerPoint PPT Presentation

1 / 16
About This Presentation
Title:

Conversational role assignment problem in multi-party dialogues

Description:

Conversational role assignment problem in multi-party dialogues Natasa Jovanovic Dennis Reidsma Rutger Rienks TKI group University of Twente – PowerPoint PPT presentation

Number of Views:51
Avg rating:3.0/5.0
Slides: 17
Provided by: nata3189
Category:

less

Transcript and Presenter's Notes

Title: Conversational role assignment problem in multi-party dialogues


1
Conversational role assignment problem in
multi-party dialogues
  • Natasa Jovanovic Dennis Reidsma
  • Rutger Rienks
  • TKI group
  • University of Twente

2
Outline
  • Research tasks at TKI
  • Interpretation of multimodal human-human
    communication in the meetings
  • Conversational Role Assignment Problem (CRAP)
  • Towards automatic addressee detection

3
A framework for multimodal interaction research
4
Multimodal annotation tool
5
Who is talking to whom?
  • CRAP as one of the main issues in multi- parity
    conversation (Traum 2003.)
  • Taxonomy of conversational roles (Herbert K.
    Clark)

speaker
addressee
side participant
bystander
all participants
all listener
eavesdropper
6
  • Our goal
  • Automatic addressee identification in small
    group discussions
  • Addressees in meeting conversations single
    participant, group of people, whole audience
  • Importance of the issue of addressing in
    multi-party dialogues

7
Addressing mechanisms
  • What are relevant sources of information for
    addressee identification in the face-to-face
    meeting conversations?
  • How does the speaker express who is the addressee
    of his utterance?
  • How can we combine all this information in order
    to determine the addressee of the utterance?

8
Sources of information
  • Speech
  • Linguistic markers
  • word classes personal pronouns, determiners in
    combination with personal pronouns, possessive
    pronouns and adjectives, indefinite pronouns,
    etc.
  • Name detection ( vocatives)
  • Dialogue acts
  • Gaze direction
  • Pointing gestures
  • Context categories(features)

9
Dialogue Acts and Addressee detection (I)
  • How many addresses may have an utterance?
  • According to dialog act theory an utterance or an
    utterance segment may have more than one
    conversational function.
  • Each DA has a addressee gt an utterance may have
    several addresses

10
Dialogue Acts and Addressee detection (II)
  • MRDA (Meeting Recorder Dialogue Acts) tag set
    for labeling multiparty face to face meetings
    (ICSI)
  • We use a huge subset of the MRDA set which is
    organized on two levels
  • Forward looking functions (FLF )
  • Backward looking functions (BLF)

11
Non-verbal features
  • Gaze
  • Contribution of the gaze to the addressee
    detection is dependent on participants location
    (visible area), utterance length, current meeting
    action
  • Turn-taking behavior and addressing behavior
  • Gesture ( pointing at a person)
  • TALK_TO (X,Y) AND POINT_TO (X,Y)
  • TALK_TO( X,Y) AND POINT_TO (X,Z) X talk to Y
    about Z

12
Context categories
  • Bunt totality of conditions that may influence
    understanding and generation of communicative
    behavior
  • Local context is an aspect of context that can be
    changed through communication
  • Context categories
  • Interaction history ( verbal and non-verbal)
  • Meeting action history
  • Spatial context (participants location,
    distance, visible area, etc. )
  • User context (name, gender, roles, etc. )

13
Towards an automatic addressee detection
  • Manual or automatic features annotation?
  • An automatic target interpreter has to deal with
    uncertainty
  • Methods
  • Rule-based method
  • Statistical method ( Bayesian networks)

14
Rule-based method
  • Processing information obtained from the
    utterance ( linguistic markers,
    vocatives, DA). The result is a list of possible
    addressees with corresponding probabilities
  • Eliminate cases where target is completely
    determined (for instance, name in vocative form)
  • Set of rules for BLF
  • Set of rules for FLF
  • Processing gaze and gesture information adding
    the additional probability values to the
    candidates

15
Meeting actions and addressee detection
  • Automatic addressee detection method can be
    applied to the whole meeting
  • Knowledge about the current meeting action as
    well as about meeting actions history may help to
    better recognize the addressee of a dialogue act.

16
Future works
  • Development of multimodal annotation tool
  • Data annotation for
  • training and evaluating statistical models
  • obtaining inputs for rule-based methods
  • New meeting scenarios for research in addressing
Write a Comment
User Comments (0)
About PowerShow.com