CSLP 2004 2004 - PowerPoint PPT Presentation

Loading...

PPT – CSLP 2004 2004 PowerPoint presentation | free to download - id: 218861-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

CSLP 2004 2004

Description:

You may let the system suggest topics or you can try your own words. ... Is miscommunication part of a transaction? What about sub tasks? ... – PowerPoint PPT presentation

Number of Views:16
Avg rating:3.0/5.0
Slides: 17
Provided by: hansd6
Learn more at: http://www.spokendialogue.dk
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CSLP 2004 2004


1
Representing ActTopicbased Dialogue Phenomena
Hans Dybkjær SpeechLogic, Prolog Development
Center A/S Laila Dybkjær NISLab, University of
Southern Denmark
2
System-user dialogue
(Opening)
  • S 0.1 PAUSE1000 Welcome to FerieKontos
    automatic guide. You may let the system
    suggest topics or you can try your own words.
    You select topics by speaking to the system.
    Choose between PAUSE200 'employee'.
    PAUSE200 On leave'. PAUSE200 'live
    abroad'. PAUSE200 Or 'stopped working' due to
    'age' or 'health'. PAUSE500 Say a topic, say
    repeat' or ask for 'more options'?
  • U 0.1 Im a student
  • S 1.1 Did you ask for Main menu
  • U 1.1 student
  • S 2.1 If you are a student with SU, you may get
    your holiday allowance without taking leave from
    the studies. But if you work besides the
    studies, you must take vacation at your work
    before you can get allowance.
  • S 2.2 If you have work, it is your employer who
    must sign your FerieKonto certificate, otherwise
    it is the social authorities in your city.
  • S 2.3 PAUSE500

(Offer)
(Request)
Start
(Check)
(Repair)
(Information)
(Information)
Success
A typical, system driven dialogue with
meta-communication
3
Dialogue, continued
  • S 2.4 Do you want to ask for more?
  • U 2.1 must I write the saturdays
  • S 3.1 From the holiday year 2002 that starts 1
    May, the vacation week is changed to 5 vacation
    days, and saturdays are thus no longer counted as
    part of the vacation.
  • S 3.2 Would you like to know more about the
    'holiday year'?
  • U 3.1 no thanks goodbye
  • S 4.1 Goodbye and thank you for calling us.
  • Event 1 nothing recorded
  • Event 2 disconnect

(Offer)
Start
(Request)
(Information)
Success
(Offer)
(Reject request)
Start
(Feedback)
(Other)
(Other)
Success
FAQ conversation has many small transactions
4
Automating annotation
  • Key concerns for commercial spoken dialogue
    systems (SDSs)
  • High transaction success rate
  • Smooth dialogue
  • Very time consuming and costly to measure
    manually
  • Manual annotation more or less only possibility
    today
  • Twostep approach towards automatic annotation
  • Annotate utterances with a basic acttopic
    structure
  • Can be automated using a parser
  • Transform acttopic patterns into transaction
    segments
  • Using rule engine

Automation only way to serious statistics
5
First step annotation
  • .u .inform T.student
  • I am a student"
  • .s .inform T.student, T.payment, T.money
  • If you are a student living from a grant
    you may get your holiday allowance while still
    studying. If you also work you need to take
    holiday to get your money.
  • .s .inform T.more
  • Is there anything else you want to ask
    about?"
  • .u .accept
  • Yes

Move Act Topic
Turn Move (of same speaker)
Context free annotation, simplistic,
6
Second step Transformation
  • Apply acttopic rules to step one annotated
    dialogues
  • Transform basic acts into composite acts

.s .inform T.pay "Payment in general .u .accept "Yes"
rule select1 _y .select Ts_a lt
_x .inform Ts_a _y .accept where
_x ! _y end rule
.u .select T.pay lt select1 .s .inform T.pay .u .accept
Formalism designed for the problem
7
Rule examples
fromplace lt place rule selectSub1 _y .select V_a lt _x .inform N_a _y .inform V_b where _a lt _b _x ! _y end rule rule success1 _y .success N_b lt _x .select T_b _y .inform Vs_a where _b in Vs_a _x ! _y end rule
.s inform N.fromplace Where does the travel start .u inform V.place Copenhagen .s inform V.fromplace .s inform N.toplace From Copenhagen, where do you want to go
Exchanges and transactions belong to different
levels
8
Dialogue phenomena investigated
  • Different acts, different topics, and different
    speakers
  • Rejecting topics
  • Differentiating topic names and topic values
  • Sets of topics
  • Patterns across turns (move sets) not using all
    moves in that turn
  • Topic relation (sub-topic) ISA (topic)
  • Metacommunication (repair, clarification, ...)
  • Multilevel rules Some rules only apply after
    match by other rules
  • Summarising feedback

Only structural phenomena propositional
contents not considered
9
Status and next steps
  • Restriction Task oriented humancomputer spoken
    dialogues
  • Declarative rewrite rules with constraints
  • Customised language close to dialogue analysis
    domain
  • Smoothness criteria not clearly defined
  • How does smoothness affect transaction counts?
  • To obtain a fully automated annotation process
  • Parse dialogues to produce basic acttopic
    annotation
  • Combine into automatic batch system
  • Need for evaluation of method
  • Test on larger number of dialogues of different
    kinds
  • Establish human coder baseline and compare

Both theoretical work and practical tools needed
10
Supplementary slides Warning you are over
time!
11
Background
  • Overthephone FAQ SDS on holiday allowance
  • General (nonperson related) questions, e.g.
  • is Saturday considered a holiday?
  • 2700 lines of grammar, 800 (full) words in
    vocabulary
  • 85 semantic concepts in input, 100 stories in
    output
  • Contractual minimum transaction success rate, but
  • transaction not clearly defined
  • no baseline from humanhuman dialogues
  • Approach to measurement
  • Transactions defined in terms of patterns of
    acttopics
  • Manually 225 test and 217 production system
    dialogues
  • Created webbased manual annotation tool

Complex domain but simple tasks
12
Problem in only identifying topics
  • Basically only distinguish between two composite
    acts
  • select continue with same topic
  • request change to new topic
  • So cannot distinguish success and failure

No success .u .inform T.phone Your phone number? .s .inform T.phone Phone number" Success .u .inform T.phone Your phone number? .s .inform T.phone Phone 48204910"
More distinction needed
13
Name and value
  • Solution distinguish
  • topic name N the mentioning of a topic
  • topic value V details about a topic

.u .inform N.phone Your phone number? .s .inform N.phone Phone number" .u .inform N.phone Your phone number? .s .inform V.phone Phone 48204910"
Simple yet powerful distinction may still be
parseable
14
More rule examples
rule answer _y .request N_b _x .inform Vs_a lt _y .inform N_b _x .inform V_b where _x ! _y end rule rule success3 _y .success N_b lt _x .request N_b _y .inform Vs_a where _b in Vs_a _x ! _y end rule
Many variations possible
15
Transaction success
  • Dialogue level task completion?
  • Works if task is welldefined and goal state
    clear
  • But many independent tasks
  • So no single clear goal state
  • Need to define transaction at subtask level
  • What constitutes a transaction?
  • Initiation and conclusion?
  • Is miscommunication part of a transaction?
  • What about subtasks?
  • Subtask level transactions may also inform on
    which parts of the system may be problematic
  • Start and end does not tell about dialogue
    smoothness

Provide users with required information
16
Smooth dialogues
  • More precise overview of problems and their
    causes and seriousness
  • Same topic may have fail and success in same call
  • Few or many repairs
  • distinction between unwanted and erroneous
    information
  • erroneous information is unacceptable (tomorrow
    is Friday, phone 36 36 00 01)
  • other information than asked for may be more or
    less serious (fax instead of phone, fax instead
    of email)
  • misunderstanding a yes for a no is usually not so
    serious (repairable) but can be a nuisance
  • Misrecognitions
  • Information blocks may contain more than asked for

You are way beyond your time frame!
About PowerShow.com