The Testing Effect: Collaborative Learning, Retrievalinduced Enhancement, and Judgments of Learning

1
The Testing Effect Collaborative Learning,
Retrieval-induced Enhancement, and Judgments of
Learning of Classroom Video Material

• Jacquelyn Cranney, Mihyun Ahn, Rachel McKinnon,,
  Sue Morris, Kaaren Watts
• School of Psychology UNSW

2
The Test Effect

• Study, or re-reading, is the obvious solution for
  learning information in order to recall it when
  required (eg final exam)
• But is re-reading material the most effective way
  to enhance later recall?
• The Test Effect
• Many laboratory-based studies have found that
  repeated testing leads to greater retention of
  information than repeated re-reading (eg Glover,
  1989 Spitzer, 1939).

3
Roediger Karpicke (2006)

• Subjects completed four learning blocks
• Block 1 all read through a prose passage
  (initial learning phase)
• Blocks 2, 3 4 either completed a free-recall
  test (RTTT) or re-read (RRRR) the passage
• final exam recall test after 5-minute interval
• RRRR had most comprehensive recall

4
Roediger Karpicke (2006)

• final exam recall test after one-week interval
• RTTT had superior recall
• Repeated reading had short-term advantage, but
  repeated testing was more effective in promoting
  long-term retention
• Proposed mechanism effortful retrieval
  processes used at the time of encoding increase
  the strength of a memory trace and increase the
  number of available retrieval routes
• Anderson et al.,1994 Butler Roediger, 2007
  Roediger Karpicke, 2006a see Karpicke
  Roediger, 2008

5
Our Studies

• Aimed to examine the generalisability of the
  testing effect in an APPLIED setting
• First-year psychology course
• What issues??

6
Judgment of Learning (JOL)

• By reflecting on the learning experience, a
  student makes a judgment about how much they have
  taken in
• Why is it important to the student?
• To determine how much additional study is
  required
• Informs confidence level in regard to knowledge
  of the information

7
Judgment of Learning (JOL)

• JOLs are often quite inaccurate!
• Resulting in
• Overconfidence/not enough study
• Underconfidence/excessive study
• Nelson Dunlosky (1991)
• Students can be highly accurate when there is a
  delay between the learning period and the JOL

8
Collaborative Educational Processes

• Group-work is becoming an educationally favoured,
  empirically-supported teaching strategy
• Particularly in undergraduate psychology (Miyake
  Shirouzu, 2006)

9
Study 1

• Test effect in applied setting?
• Delayed JOLs accurate in applied setting?
• Collaborative learning situation enhances the
  testing effect?
• Material would be on final exam

10
METHOD

• Phase One
• 8-minute video on Psychobiology - the topic
  concurrently being presented in lectures
• Semi-random assignment to condition (based on
  tutorial group) determined the post-video
  activity
• Group Quiz (GQ)
• Individual Quiz (IQ)
• Highlighting (H)
• Control Activity (C)

11
Conditions

• Group Quiz (GQ)
• Groups of 4-5 students
• As a group completed 10-question quiz about video
  material (with group discussion)
• Received feedback
• Individual Quiz (IQ)
• Same quiz, but completed individually (without
  discussion)
• Received feedback

12
Conditions

• Highlighting (H) re-study, re-read
• Read through a transcript of video
• Highlighted information they considered important
• Control Activity (C)
• Did not re-engage with video information
• Students watched an unrelated video

13
METHOD

• Phase Two (ONE week after Phase One)
• Unexpected pop-quiz in tutorial
• Questions
• Based on Video
• Some old presented in Phase 1 Quiz
• Some new presented only in Phase 2 Quiz
• Based on Lectures
• Prior to completing quiz
• Judged the grade they thought they might receive
  on pop-quiz (JOL)

14
RESULTS

• Phase One Quiz
• Group scores were higher than individual scores
• Group average 94
• Individual average 47
• Likely due to the benefit of pooled information

15
RESULTS

• Phase Two Quiz
• Old Items

16
RESULTS

• Lenient Criterion Test Effect v
• Phase 1 quiz (Group Quiz Individual Quiz)
  better than no Phase 1 quiz (Highlight Control)
• Individual Quiz better than Control

17
RESULTS

• Strict Criterion Test Effect
• Individual Quiz not better than Highlight

18
RESULTS

• Collaborative Learning Benefit v
• Group quiz better than Individual quiz
• ie effect maintained following one week interval
• Reflects impact of good performance in Phase 1 on
  strength of Test Effect mechanisms?

NEW ITEMS?
19
RESULTS

• Phase Two Quiz
• New Items

20
RESULTS

• The Test Effect did not generalise to previously
  unseen test items (Duchastel, 1981)
• Individual Quiz not better than Highlight
• Individual Quiz not better than Control

21
RESULTS

• Collaborative Learning Effect v
• Group Test Effect
• Group Quiz better than Individual Quiz
• Broader engagement with the video information as
  a result of group discussion in Phase 1?

22
RESULTS

• JOL
• Participants late predicted grades (JOLs)
  correlated strongly with their actual grades
• NO group differences

23
Conclusions

• Our study demonstrated in an APPLIED setting
• The Testing Effect (lenient criterion)
• But did not generalise to related new questions
• The benefits of Collaborative Testing
• To enhance accuracy during initial test
• To improve retention for old and new
  questions
• Accurate delayed JOL
• BUTdid NOT replicate individual strict criterion
  test effect one focus of Study 2

24
Study 2

• Study 1 - an initial test (Phase 1) enhanced
  recall of the old material on a later test (Phase
  2)
• However, taking an initial test may -
• Enhance later memory for new, related material
  retrieval-induced facilitation (Chan et al.,
  2006), or
• Inhibit later memory for new, related material
  retrieval-induced forgetting (Anderson, 2003)

25
Study 2

• Objectives
• to replicate the individual strict-criterion test
  effect
• to determine whether recall for nontested-related
  material was enhanced or inhibited in a real
  educational setting

26
METHOD

• Phase One
• N 218 first-year psychology students
• Video presentation was identical to Study 1
• Semi-random assignment to condition (based on
  tutorial group) determined the post-video
  activity (3 conditions)
• Individual Quiz (Q)
• Highlighting (H)
• Control (C)

27
ConditionsPhase 1

• Quiz (Q)
• Individuals completed a 10-question quiz (cued
  recall items) about the video material
• Highlight (H) re-read, re-study
• Read through 10 summary points of video material
• Highlighted/underlined key points
• Control (C)
• Did not re-engage with video information

28
METHOD

• Phase Two (one week after Phase One)
• Unexpected pop-quiz in tutorial
• Questions
• Based on Video
• Same presented in Phase 1 Quiz or as summary
  points
• Related presented in Phase 2 pop-quiz only
• All based on video material

29
Study 2

• Hypotheses
• Phase Two Recall Test Effect for Same Material
• Lenient Criterion Quiz would do better than the
  Control
• Strict Criterion Quiz would do better than the
  Highlight

30
Study 2

• Hypotheses
• Phase Two Recall retrieval-induced facilitation
  vs. forgetting for Related Material
• facilitation - Quiz would do better than the
  Highlight and Control, or
• RIF Quiz would do worse than the Highlight and
  Control

31
RESULTS

• Test Effect Same Material
• Lenient Criterion Test Effect v
• Quiz better than Control
• Strict Criterion Test Effect v
• Quiz better than Highlight

32
RESULTS

• Facilitation vs RIF Related (Different)
  Material
• Quiz better than Highlight
• Quiz better than Control
• Highlight not better than Control

33
RESULTS

• Test Effect
• Lenient and Strict criteria v
• Facilitation vs RIF
• Retrieval-Induced Facilitation hypothesis v
  consistent with Chan et al. (2006)
• Retrieval-Induced Forgetting hypothesis X
• Suggests elaborative retrieval processes activate
  schemas and semantic processes for related
  information as well as for the target/ same
  information

34
RESULTS

• Initial testing enhanced recall for previously
  presented material
• Initial testing enhanced, rather than inhibited,
  recall for previously untested, related material,
  consistent with Chan et al. (2006)

35
Relevance for Education

• Highlights benefits of testing (cf re-reading
  material) as an assessment and learning tool
  mechanisms?
• Testing on some of the material may enhance
  (rather than interfere with) later recall of
  related materialRIF may only occur with short
  test-re-test intervals (Carroll, 2007)
• Need more research on effects of feedback
  (McDaniel et al., 2007) mechanisms?
• Collaborative testing facilitates learning
  mechanisms? (Basden, Basden, Henry, 2000Weldon
  Bellinger, 1997 )
• Judging ones own learning for study planning and
  motivational purposes is relatively accurate when
  delayed from the learning experience (cf Karpicke
  Roediger, 2008)

36
Acknowledgements

• 2006 and 2007 UNSW PSYC1011 students and tutors
• Funding from UNSW School of Psychology

37
Some References

• Anderson, M. C., (2003). Rethinking interference
  theory Executive control and the mechanisms of
  forgetting. Journal of Memory Language, 49,
  415-445.
• Chan, C. K., McDermott, B., Roediger III, H. L.
  (2006). Retrieval-induced facilitation Initially
  nontested material can benefit from prior testing
  of related material. Journal of Experimental
  Psychology General, 135(4), 553-571.
• Duchastel, P. C. (1981). Retention of prose
  following testing with different types of tests.
  Contemporary Educational Psychology, 6, 217-226.
• Glover, J. A. (1989). The testing phenomenon
  Not gone but nearly forgotten. Journal of
  Educational Psychology, 81, 329-399.
• Miyake, N., Shirouzzu, H. (2006). A
  collaborative approach to teaching cognitive
  science to undergraduates The learning sciences
  as a means to study and enhance college student
  learning. Psychologia An International Journal
  of Psychology in the Orient, 49(2), 101-113.
• Nelson, T. O., Dunlosky, J. (1991). When
  peoples judgments of learning (JOLs) are
  extremely accurate at predicting subsequent
  recall The Delayed-JOL Effect. Psychological
  Science, 2(4), 267-270.
• Roediger, H. L., III, Karpicke, J. D. (2006).
  Test-enhanced learning Taking memory tests
  improves long-term retention. Psychological
  Science, 17(3), 249-255.
• Spitzer, H. F. (1939). Studies in retention.
  Journal of Educational Psychology, 30, 641-656.