CSLP 2004 2004 - PowerPoint PPT Presentation


PPT – CSLP 2004 2004 PowerPoint presentation | free to download - id: 218861-ZDc1Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

CSLP 2004 2004


You may let the system suggest topics or you can try your own words. ... Is miscommunication part of a transaction? What about sub tasks? ... – PowerPoint PPT presentation

Number of Views:16
Avg rating:3.0/5.0
Slides: 17
Provided by: hansd6
Learn more at: http://www.spokendialogue.dk


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CSLP 2004 2004

Representing ActTopicbased Dialogue Phenomena
Hans Dybkjær SpeechLogic, Prolog Development
Center A/S Laila Dybkjær NISLab, University of
Southern Denmark
System-user dialogue
  • S 0.1 PAUSE1000 Welcome to FerieKontos
    automatic guide. You may let the system
    suggest topics or you can try your own words.
    You select topics by speaking to the system.
    Choose between PAUSE200 'employee'.
    PAUSE200 On leave'. PAUSE200 'live
    abroad'. PAUSE200 Or 'stopped working' due to
    'age' or 'health'. PAUSE500 Say a topic, say
    repeat' or ask for 'more options'?
  • U 0.1 Im a student
  • S 1.1 Did you ask for Main menu
  • U 1.1 student
  • S 2.1 If you are a student with SU, you may get
    your holiday allowance without taking leave from
    the studies. But if you work besides the
    studies, you must take vacation at your work
    before you can get allowance.
  • S 2.2 If you have work, it is your employer who
    must sign your FerieKonto certificate, otherwise
    it is the social authorities in your city.
  • S 2.3 PAUSE500

A typical, system driven dialogue with
Dialogue, continued
  • S 2.4 Do you want to ask for more?
  • U 2.1 must I write the saturdays
  • S 3.1 From the holiday year 2002 that starts 1
    May, the vacation week is changed to 5 vacation
    days, and saturdays are thus no longer counted as
    part of the vacation.
  • S 3.2 Would you like to know more about the
    'holiday year'?
  • U 3.1 no thanks goodbye
  • S 4.1 Goodbye and thank you for calling us.
  • Event 1 nothing recorded
  • Event 2 disconnect

(Reject request)
FAQ conversation has many small transactions
Automating annotation
  • Key concerns for commercial spoken dialogue
    systems (SDSs)
  • High transaction success rate
  • Smooth dialogue
  • Very time consuming and costly to measure
  • Manual annotation more or less only possibility
  • Twostep approach towards automatic annotation
  • Annotate utterances with a basic acttopic
  • Can be automated using a parser
  • Transform acttopic patterns into transaction
  • Using rule engine

Automation only way to serious statistics
First step annotation
  • .u .inform T.student
  • I am a student"
  • .s .inform T.student, T.payment, T.money
  • If you are a student living from a grant
    you may get your holiday allowance while still
    studying. If you also work you need to take
    holiday to get your money.
  • .s .inform T.more
  • Is there anything else you want to ask
  • .u .accept
  • Yes

Move Act Topic
Turn Move (of same speaker)
Context free annotation, simplistic,
Second step Transformation
  • Apply acttopic rules to step one annotated
  • Transform basic acts into composite acts

.s .inform T.pay "Payment in general .u .accept "Yes"
rule select1 _y .select Ts_a lt
_x .inform Ts_a _y .accept where
_x ! _y end rule
.u .select T.pay lt select1 .s .inform T.pay .u .accept
Formalism designed for the problem
Rule examples
fromplace lt place rule selectSub1 _y .select V_a lt _x .inform N_a _y .inform V_b where _a lt _b _x ! _y end rule rule success1 _y .success N_b lt _x .select T_b _y .inform Vs_a where _b in Vs_a _x ! _y end rule
.s inform N.fromplace Where does the travel start .u inform V.place Copenhagen .s inform V.fromplace .s inform N.toplace From Copenhagen, where do you want to go
Exchanges and transactions belong to different
Dialogue phenomena investigated
  • Different acts, different topics, and different
  • Rejecting topics
  • Differentiating topic names and topic values
  • Sets of topics
  • Patterns across turns (move sets) not using all
    moves in that turn
  • Topic relation (sub-topic) ISA (topic)
  • Metacommunication (repair, clarification, ...)
  • Multilevel rules Some rules only apply after
    match by other rules
  • Summarising feedback

Only structural phenomena propositional
contents not considered
Status and next steps
  • Restriction Task oriented humancomputer spoken
  • Declarative rewrite rules with constraints
  • Customised language close to dialogue analysis
  • Smoothness criteria not clearly defined
  • How does smoothness affect transaction counts?
  • To obtain a fully automated annotation process
  • Parse dialogues to produce basic acttopic
  • Combine into automatic batch system
  • Need for evaluation of method
  • Test on larger number of dialogues of different
  • Establish human coder baseline and compare

Both theoretical work and practical tools needed
Supplementary slides Warning you are over
  • Overthephone FAQ SDS on holiday allowance
  • General (nonperson related) questions, e.g.
  • is Saturday considered a holiday?
  • 2700 lines of grammar, 800 (full) words in
  • 85 semantic concepts in input, 100 stories in
  • Contractual minimum transaction success rate, but
  • transaction not clearly defined
  • no baseline from humanhuman dialogues
  • Approach to measurement
  • Transactions defined in terms of patterns of
  • Manually 225 test and 217 production system
  • Created webbased manual annotation tool

Complex domain but simple tasks
Problem in only identifying topics
  • Basically only distinguish between two composite
  • select continue with same topic
  • request change to new topic
  • So cannot distinguish success and failure

No success .u .inform T.phone Your phone number? .s .inform T.phone Phone number" Success .u .inform T.phone Your phone number? .s .inform T.phone Phone 48204910"
More distinction needed
Name and value
  • Solution distinguish
  • topic name N the mentioning of a topic
  • topic value V details about a topic

.u .inform N.phone Your phone number? .s .inform N.phone Phone number" .u .inform N.phone Your phone number? .s .inform V.phone Phone 48204910"
Simple yet powerful distinction may still be
More rule examples
rule answer _y .request N_b _x .inform Vs_a lt _y .inform N_b _x .inform V_b where _x ! _y end rule rule success3 _y .success N_b lt _x .request N_b _y .inform Vs_a where _b in Vs_a _x ! _y end rule
Many variations possible
Transaction success
  • Dialogue level task completion?
  • Works if task is welldefined and goal state
  • But many independent tasks
  • So no single clear goal state
  • Need to define transaction at subtask level
  • What constitutes a transaction?
  • Initiation and conclusion?
  • Is miscommunication part of a transaction?
  • What about subtasks?
  • Subtask level transactions may also inform on
    which parts of the system may be problematic
  • Start and end does not tell about dialogue

Provide users with required information
Smooth dialogues
  • More precise overview of problems and their
    causes and seriousness
  • Same topic may have fail and success in same call
  • Few or many repairs
  • distinction between unwanted and erroneous
  • erroneous information is unacceptable (tomorrow
    is Friday, phone 36 36 00 01)
  • other information than asked for may be more or
    less serious (fax instead of phone, fax instead
    of email)
  • misunderstanding a yes for a no is usually not so
    serious (repairable) but can be a nuisance
  • Misrecognitions
  • Information blocks may contain more than asked for

You are way beyond your time frame!
About PowerShow.com