Using Information State to Improve Dialog Move Identification in a Spoken Dialog System Hua Ai, Antonio Roque, Anton Leuski, David Traum ISSP, University of Pittsburgh; ICT, University of Southern California

1
Using Information State to Improve Dialog Move
Identification in a Spoken Dialog SystemHua Ai,
Antonio Roque, Anton Leuski, David TraumISSP,
University of Pittsburgh ICT, University of
Southern California
1
2
2
2
1
2
System Overview
Background

• Different types of contextual information is used
  in different ways
• Use other words surrounding the utterance of
  interest as context (e.g., Taylor et al., 1998)
• Use dialog state as context (e.g., Bohus and
  Rudnicky, 2003)
• Use dialog moves and dialog parameters as context
  in dialog managers which employ information state
  approach (e.g., Traum and Larsson, 2003)
• Use context to provide expectations of likely
  utterances (Smith and Richard Hipp, 1994)

Radiobot-CFF is a dialog agent built for Call for
Fire (CFF) radio dialogs. In a CFF, a Forward
Observer (FO) team (played by trainee) identifies
a target to be attacked by indirect artillery
fire. A Fire Direction Center (FDC) (played by
computer dialog agent) sends an artillery
mission. The FO and the FDC coordinate the
attack.
Our Approach
Example Dialog
Use 9 information state features as contextual
information treat the contextual information as
features that interpreter can use. Examples of
contextual features Has target location, Phase,
Method of fire, etc.
FO steel one niner this is gator niner one
adjust fire polar over FDC gator nine one this
is steel one nine adjust fire polar out
Experiments
Predicted Classes
Results
Results shown here are using the best model CRF
tagger trained on ASR output. Refer to paper for
other results.
Hypothesis
Corpus Evaluation Tagging Accuracies
Online Evaluation Tagging Accuracies
Adding contextual information can improve the
performance of the dialog move and dialog
parameter taggers and identify ASR quality
(c-correct, s-substitute, i-inserted, d-deleted).
Conclusion

1. Using contextual information improves the Dialog
   Move and Dialog Parameter Taggers performance
2. It is possible to recover ASR errors using a
   tagger with contextual information
3. Conditional Random Field Tagger outperforms J48
   Decision Tree on DM, DP and word prediction
4. Taggers trained on ASR outputs outperform
   taggers trained on transcriptions

Transcription steel one niner this is gator niner one over
ASR output still one niner is gator niner a one over
Quality s c c d c c c i c c
DM ID ID ID ID ID ID ID NULL ID None
DP fdc_id fdc_id fdc_ic none none fo_id fo_id NULL fo_id nonc