An Introduction to TuTalk: Developing Dialogue Agents for Learning Studies

1
An Introduction to TuTalk Developing Dialogue
Agents for Learning Studies

• Pamela Jordan
• University of Pittsburgh
• Learning Research and Development Center

2
Dialogue in Learning Applications

• Language practice
• Peer collaborations
• Short answer tutorial dialogue
• AutoTutor
• Computer literacy qualitative physics
• More effective than reading a textbook
• Circsim-Tutor
• In use in medical school classrooms
• More effective than problem solving alone
• TuTalk
• Quantitative and qualitative physics,
  thermodynamics, program design, conceptual math
• More effective than reading a textbook
• In some situations as effective as a human tutor

3
Circsim-Tutor
4
What is TuTalk?

• Dialogue system construction tools for learning
  applications
• Authoring tools for creating, testing and
  maintaining an artificial dialogue partner/agent
• Dialogue agent server that
• Conducts dialogues with multiple students and are
  either
• agent-led (system initiates all topics) or
• mixed-initiative dialogues (student or system can
  initiate topics)
• Provides communication protocol for connecting to
  application specific interface
• Provides experiment management tools

5
Finite state dialogue management
gravitational,
ltanything elsegt
runners
6
Example dialogue
Subdialogue
7
Integrating TuTalk

• No interface building tools but several examples
  provided with server software just to show how to
  do it
• Can integrate it into another tool or create your
  own interface (e.g. in CTAT for summer school 06
  indicated a URL but now requires a special secure
  communication protocol)
• Once integrated into another tool, others can use
  too

8
TuTalk History

• Andes-Atlas for tutoring quantitative physics
  problem solving (3 experiments)
• Why2-Atlas for tutoring qualitative physics
  problem solving (3 experiments)
• ITSpoke spoken language interface to Why2-Atlas
  (2 experiments)
• Reflective Follow-up dialogues following
  quantitative physics problem solving (2
  experiments)
• CycleTalk for tutoring thermodynamics design
  problems (1 experiment)
• ProPL for tutoring program design (1 experiment)

9
ProPL
10
When are short answer dialogues
appropriate/inappropriate?

• Appropriate for
• practicing some dialogue skills
• conceptual discussions
• scaffolding problem solving
• identifying addressing gaps in student
  understanding only as needed (hints, examples)
• Not appropriate for
• assessing deep understanding
• addressing grammar problems in language
• content delivery printed text is more efficient
• student-only initiative (use CTAT for that)

11
What do you have to do to create a TuTalk
dialogue agent?

• Write domain content in form of natural language
  dialogue turns (e.g. elicit or tell)
• Write expected student responses (short answer
  responses)
• Write subdialogues for expected student responses
  that are
• Vague
• Incorrect
• Overly specific

12
Example methodology

• Write an ideal dialogue in English for a topic or
  knowledge component
• Go back through it and think about possible
  alternative responses
• For each alternative response that should react
  to differently write a subdialogue

13
Authoring definitions tutoring perspective

• A collection of dialogues that make up an
  experimental condition is called a
  script/scenario
• A dialogue covers a goal (aka topic)
• One goal/topic can have alternative dialogues an
  instance of a dialogue for a goal is called a
  template
• A dialogue has one or more tutor turns called an
  initiation
• An initiation can have an expected student
  response
• An initiation response, or initiation with no
  expected response is called a step
• A set of alternative phrasings for an initiation
  or response is called a concept

14
Example template for a dialogue covering a goal
(abstract)
Goal name
Goal select-appetizer step enthuse_about_appeti
zers step ask_share_appetizer
agree_to_share_appetizer skip_appetizer
abort, ask-soup unknown abort,
loose-temper step agree-on-appetizer
Concept to realize or recognize
initiation
possible responses
Response action push to subdialogue for this goal
Push to subdialogue for this goal
15
Authoring a dialogue with subdialogues
16
Authoring a subdialogue
17
Extending concept definitions
18
Previewing authored dialogues
19
Testing with dialogue agent server
20
Authoring, Previewing and Testing Demo
21
Past Summer School TuTalk projects

• Language tutoring
• Coaching military trainees to follow one required
  communications protocol
• Coaching student is proper use of two Chinese
  lexical items that depend on context
• Conceptual tutoring
• Coaching elementary school students in
  qualitative reasoning skills
• Coaching students on loop constructs in
  programming
• Coaching students in the solution of monomials

22
Tuesday TuTalk Offerings

• See http//andes3.lrdc.pitt.edu/TuTalk/corpora/
• The Methodology and Basics of Authoring TuTalk
  Dialogue Agents (in Track room, not in lab)
• Dialogue authoring methodologies  
• Advice/findings on effective learning dialogues
• Review and expand on basic features of TuTalk
• Create a simple TuTalk Dialogue Agent in the
  project room
• Do section 3.3 of TuTalk Authoring Interface
  Users Guide (you can do sections 3.1 and 3.2
  first if you prefer)
• Well be there to answer questions and help you
• Advanced TuTalk Dialogue Agents (in Track room)
• Learn about the TuTalk server and its support
  tools
• Explore additional authoring features (e.g.
  controlling automatic response feedback, looping,
  optional steps)

23
TuTalk Team

• Authoring tools
• Carolyn Rosé
• Yue Cui (Jenny)
• Rohit Kumar
• Dialogue server
• Pam Jordan
• Brian Hall (Moses)