An analysis of generative dialogue patterns across interactive learning environments: Explanation, e

1
An analysis of generative dialogue patterns
across interactive learning environments
Explanation, elaboration, and co-construction

• Robert G.M Hausmann
• Pittsburgh Science of Learning Center (PSLC)
• Learning Research and Development Center
• University of Pittsburgh

2
Top-level Goal
Cognitive
X
Social
Deep Learning
Affective
3
Outline

• Introduction
• Thesis
• Definitions
• Methodology
• Evidence
• Individual learning
• Human tutoring (novice expert)
• Peer collaboration
• Observing Tutorial Dialogs Collaboratively
• Discussion
• Integration with serious games

4
Thesis

• Part 1 There are several paths toward learning.
• Some paths are better suited for the acquisition
  of different representations (Nokes Ohlsson,
  2005).
• Part 2 Generative interactions produce deep
  learning.
• Increasing the probability that a generative
  interaction occurs should increase the
  probability of robust learning.
• Part 3 Different interactive learning
  environments differentially support generative
  interactions.
• Learning with understanding vs. performance
  orientation (Schauble, 1990 Schauble Glaser,
  1990).

5
Definitions

• Learning
• Revise of mental model
• Application of conceptual knowledge
• Reduction of errors during the acquisition of
  procedural knowledge
• Interaction
• Dialog situation-relevant response that occurs
  between two or more agents (human or computer).
• Monolog statements uttered out loud that reveal
  the individuals understanding processes.
• Generative
• Produce inferences (ex the lungs are the site of
  oxygenation)
• Apply knowledge to a problem (ex applying
  Newtons second law)

6
Methodology high-level description

• Collect transcribe a corpus of learning
  interactions
• Categorize statements (Chi, 1997)
• Assess learning at multiple levels of depth
• Correlate statements with shallow and deep
  learning
• Follow-up Study Experimentally manipulate
  interaction type (goto 1)

7
Study 1 Self-explaining vs. Paraphrasing

• Procedure
• Domain Circulatory system
• Pretest
• Prompting intervention (41.1 min.)
• Posttest
• Participants
• University of Pittsburgh undergraduates (N 40)
• Course credit
• Research Questions
• Can a computer interface use generic prompts to
  inspire students to self-explain?
• If so, what is the effect on learning?

8
Textual Materials
Generic Tutorial Prompts
Self-explanation Field
9
Results Self-explanation Frequency (Exp. 2)
Source Hausmann and Chi (2002)
10
Results Correlations with Learning (Exp. 2)
Source Hausmann and Chi (2002)
11
Study 2 Coverage vs. Generation

• Method
• Domain Electrostatics
• Procedure Alternate between problem-solving and
  example-studying (110 min.)
• Design Example (complete vs. incomplete) x Study
  Strategy (self-explain vs. paraphrase)
• Participants
• U.S. Naval Academy midshipmen (N 104)
• Course credit
• Research Questions
• Does learning depend on the type of processing or
  the completeness of the examples?
• Does prompting for self-explaining also work in
  the classroom?

12
Define Variables
Draw Coordinates
Write Equations
Draw Vectors
Tutorial Hints
13
Method Timeline
Problem1
Problem2
Problem3
Problem4
14
Results Bottom-out Help
Source Hausmann and VanLehn (in prep)
15
Results Assistance Score
Source Hausmann and VanLehn (in prep)
16
Study 3 Novice, Human Tutoring

• Procedure
• Domain circulatory system
• Pretest (w/ textbook)
• Intervention (120 min.)
• Posttest
• Participants
• Tutors Nursing Students (N 11)
• Tutee Eighth-Grade Students (N 11)
• Paid volunteers
• Research Questions
• How do novice tutors naturally interact with
  students?
• Can tutors be trained to interact in specific
  way, and what impact does alternative tutorial
  dialogs have on learning?

17
Results Learning

• Knowledge pieces
• Study 1 Pretest 13 Posttest 46, p lt .001
• Study 2 Pretest 22 Posttest 45, p lt .001
• Mental model
• Study 1 Pretest 0 Posttest 73
• Study 2 Pretest 0 Posttest 64

Source Chi, Siler, Jeong, Yamauchi, and Hausmann
(2001)
18
Results Learning

• Question-answering

Source Chi, Siler, Jeong, Yamauchi, and Hausmann
(2001)
19
Results (Exp. 1) Types of tutor moves and
student responses
Source Chi, Siler, Jeong, Yamauchi, and Hausmann
(2001)
20
Results (Exp. 2) Types of tutor moves and
student responses
Source Chi, Siler, Jeong, Yamauchi, and Hausmann
(2001)
21
Study 4 Comparison of Multiple Interactive
Learning Environments

• Procedure
• Domain Newtonian Mechanics
• Pretest (w/ textbook)
• Instructional Intervention (next 5 slides)
• Posttest (w/o textbook)
• Participants
• University of Pittsburgh undergraduates (N 70)
• Paid volunteers
• Research Questions
• What types of interactions are related to
  learning from tutoring and collaboratively
  observing tutoring?
• Why do peers learn from collaboration?

22
Experimental Design

• Intervention 1 Tutoring
• Learning Resource Expert human tutor
• One student (n 10 video tapes)

Fma m3kg
Tutoring session 1
23
Experimental Design

• Intervention 2 Observing Collaboratively
• Learning Resource Peer Videotape
• Two students (n 10 yoked design)

Fma m1m2?
Tutoring session 1
24
Experimental Design

• Intervention 3 Observing Alone
• Learning Resource Videotape
• One student (n 10 yoked design)

Tutoring session 1
25
Experimental Design

• Intervention 4 Collaborating
• Learning Resource Peer Text
• Two students (n 10)

Fxmax Fx7kgx2.6m/s2
26
Experimental Design

• Intervention 5 Studying Alone
• Learning Resource Text
• One student (n 10)

Fymg Fy2kgx9.8m/s2
27
Results Condition Differences
Pre
Post
Source Chi, Roy, and Hausmann (accepted)
28
Results Interaction Analysis
Source Chi, Roy, and Hausmann (accepted)
29
Results Condition Differences
Pre
Post
Source Chi, Roy, and Hausmann (accepted)
30
Results Collaborative Dyads
Source Hausmann, Chi, and Roy (2002)
31
Results Collaborative Dyads
Source Hausmann, Chi, and Roy (2002)
32
Results Collaborative Dyads
Source Hausmann, Chi, and Roy (2002)
33
Results Collaborative Dyads
Source Hausmann, Chi, and Roy (2002)
34
Study 5 Interaction Training

• Procedure
• Domain Conceptual Engineering (bridge design)
• Pretest (w/ textbook)
• Intervention (5 min.)
• Problem-solving Task (30 min.)
• Posttest
• Participants
• University of Pittsburgh undergraduates (N 136)
• Course credit
• Research Questions
• Can undergraduate dyads be trained to interact
  effectively (i.e., co-construct)?
• What effect do certain dialog types have on
  problem solving and learning?

35
Fabrication Cost
Modify Properties
Member List
36
Fabrication Cost
Modify Properties
Member List
37
(No Transcript)
38
Color-coded Feedback
39
Results Manipulation Check
Source Hausmann (2006)
40
Results Problem Solving
Source Hausmann (2006)
41
Results Learning
Source Hausmann (2006)
42
Summary

• Individual Learning (Studies 1 2)
• Paraphrasing gt shallow learning
• Self-explaining gt deep learning
• Human Tutoring (Studies 3 4)
• Listening to tutor explain gt shallow learning
• Receiving scaffolding gt deep learning
• Reflective comments gt deep learning
• Peer collaboration (Studies 4 5)
• Listening to peer explain gt shallow learning
• Giving an explanation to peer gt deep learning
• Co-constructing knowledge gt deep learning
• Observing Tutoring Collaboratively (Studies 4)
• Observing tutor explain is not correlated with
  deep learning
• Observing student receive scaffolding gt deep
  learning

43
Integration with Serious Games

• What is the implication of these results for the
  design of serious games?
• How can a game inspire explanation, elaboration,
  or even co-construction?

44
Acknowledgements

• Funding Agencies
• Support _at_ LRDC
• Gary Wild
• Shari Kubitz
• Eric Fussenegger

• Advisors
• Michelene T.H. Chi
• Kurt VanLehn
• Physics Instructors
• Donald J. Treacy, USNA
• Robert N. Shelby, USNA
• Other Influences
• Mark McGregor
• Marguerite Roy
• Rod Roscoe
• Programmers
• Anders Weinstein
• Brett van de Sande

45
References

• Study 1 Hausmann, R.G.M. Chi, M.T.H. (2002)
  Can a computer interface support self-explaining?
  Cognitive Technology, 7(1), 4-15.
• Study 2 Hausmann, R.G.M. VanLehn (in prep).
  The effect of generation on robust learning.
• Study 3 Chi, M.T.H., Siler, S., Jeong, H.,
  Yamauchi, T., Hausmann, R.G. (2001). Learning
  from human tutoring. Cognitive Science, 25(4),
  471-533.
• Study 4a Chi, M.T.H., Roy, M., Hausmann
  (accepted). Observing Tutorial Dialogues
  Collaboratively Insights about Tutoring
  Effectiveness from Vicarious Learning, Cognitive
  Science, x, p. xxx-xxx.
• Study 4b Hausmann, R.G.M., Chi, M.T.H., Roy,
  M. (2004) Learning from collaborative problem
  solving An analysis of three hypothesized
  mechanisms. 26nd Annual Meeting of the Cognitive
  Science Conference, Chicago, IL.
• Study 5 Hausmann, R. G. M. (2006). Why do
  elaborative dialogs lead to effective problem
  solving and deep learning? Poster presented at
  the 28th Annual Meeting of the Cognitive Science
  Conference, Vancouver, Canada.

46
Inferential Mechanisms

• Simulation of a mental model (Norman, 1983)
• Category membership (Chi, Hutchinson, Robin,
  1989)
• Analogical reasoning (Markman, 1997)
• Integration of the situation and text model
  (Graesser, Singer, Trabasso, 1994)
• Logical reasoning (Rips, 1990)
• Self-explanation (Chi, Bassok, Lewis, Reimann,
  Glaser, 1989)

47
Cognitive Social Factors

• Different types of interaction lead to different
  representations
• Non-generative interactions lead to shallow
  learning.
• Definition does not modify material in any
  meaningful way.
• Generative interactions lead to deep learning.
• Definition significantly modifies the material
  in a meaningful way.