Using Video Analysis to Develop Shared WordstoImages Language for Describing Science Teaching

1
Using Video Analysis to Develop Shared
Words-to-Images Language for Describing Science
Teaching
Kathleen J. Roth November 2008 DRK-12 PI Meeting
2
Overview of session

• PART 1 About the TWIST synthesis project (15
  min)
• Why do WE need to tie words to images of
  science teaching?
• PART II Engaging in the work of the TWIST
  project (45 min)
• PART III Discussion (15 min)
• How important is it for the science education
  community to have a shared words-to-images
  language for describing science teaching?
• Do you think its possible for the science
  education research community to develop such
  consensus?
• What will increase the value of such an effort?
• What are the pitfalls?

3
PART 1 About the TWIST Project

• The Problem
• The Research Question
• The Goals
• The Strategy

4
The Problem Lack of a shared words-to-images
language for describing science teaching

• Reform Teaching Observation Protocol (Piburn and
  Sawada, 2000)
• Inquiry Science Observation Protocol (Minner et
  al., Education Development Center)
• TIMSS Video Study Coding Manual (Roth et al.,
  LessonLab Research Institute, 2006)
• Looking Inside the Classroom Observation Protocol
  (Weiss et al., Horizon Research, 2003)
• CRESST/Rand Science Observation Rating Guide
  (Borko and Stecher, 2006)
• Analysis of Inquiry Rubric (Adams et al., 2007)
• Science Classroom Observation Rubric (Ashbacher,
  Pine, and Bower, 1999)
• Science Classroom Observation Guide (Chicago Math
  and Science Initiative, 2008)
• Process oriented teaching in physics instruction
  (Seidel and Kobarg2007)
• Enacting reform-based science (Schneider,
  Krajcik, and Blumenfeld, 2005)

5
The Problem Lack of a shared
words-to-images language for describing science
teaching

• Interpretation of findings problem
• Limited impact problem
• Communication to practitioners and policymakers
  problem

6
The Problem Why is a shared language important
to address now?

• Value of standards in history of science
• Strong and diverse research base
• National interest in identifying teaching quality
• Video and computer technology
• Growing research interest using video

7
The Research Question

• Can the science education research community take
  advantage of video technology and a strong and
  diverse set of existing research tools for
  describing science teaching to develop a
  consensus research language that is image-based?

8
The Goals

• To develop shared words-to-images language for
  describing 5-10 aspects of science teaching
• To test the effectiveness of these
  words-to-images research tools in a national
  field study
• To produce a Guide to Video Analysis of
  ScienceTeaching (coding manual with video clips
  for researchers to use in establishing
  reliability, inter-rater agreement)

9
The Strategy

• Literature Review
• Expert panel (9)
• Consensus Panel (20)
• National field study participants (100)

10
The Strategy

• Expert Panel will
• Select 5-10 features of science teaching to be
  addressed in this study based on
• Links to student learning
• Potential for wide use in research
• Shared coding experiences in a face-to-face
  setting
• Develop a consensus strategy for coding selected
  video clips by
• Individually coding video examples using existing
  coding strategies (prior to meeting)
• Examining together issues that arise from looking
  at coding differences (face-to-face)
• Revising coding strategy
• Send for review to Consensus Panel

11
PART 2 Engaging in the work of the TWIST Project

• Expert Panel will
• Select 5-10 features of science teaching to be
  addressed in this study based on
• Potential for wide use in research
• Links to student learning
• Shared coding experiences in a face-to-face
  setting
• Develop a consensus strategy for coding selected
  video clips by
• Individually coding video examples using existing
  coding strategy (prior to meeting)
• Examining together issues that arise from looking
  at coding differences (face-to-face)
• Revising coding strategy
• Send for review to Consensus Panel

12
Nominated feature to analyze

• Coherence of science content development in
  lessons

13
Why coherence?TIMSS Video Study of 8th Grade
Science Teaching
14
Why coherence? Results form the STeLLA PD program
15
Why coherence?Results from the STeLLA Study

• Content Storyline Strategies
• One main learning goal
• Set purpose with goal statement or focus question
• Select activities matched to learning goal
• Link science content ideas and activities
• Link content ideas to other content ideas
• Content representations matched to learning goal
• Summarize and synthesize
• Sequence key ideas and activities appropriately

Student Science Learning

• Student Thinking Strategies
• Elicit student ideas
• Ask probing/challenge questions
• Engage students in interpreting and reasoning
  about data and observations
• Engage students in using and applying new ideas
  in a variety of ways and contexts
• Engage students in making connections through
  synthesizing and summarizing work

16
How to code?

• Identifying specific indicators of coherence
• Judgment level codes Is there coherent
  development of science ideas, with a strong
  science content storyline?
• 1 no
• 2 poor
• 3 fair
• 4 good
• 5 excellent

17
Looking for indicators of coherence in videos of
science teaching
18
Analyzing Eighth-grade TIMSS Video Study Lessons
U.S. and Japan

• Do the teachers build coherent science content
  storylines?
• Are students challenged to link science ideas to
  activities or are they simply following
  procedures?

19
US Clip 1
20
Japan Clip 1
21
Japan Clip 2
22
Compare coding

• Which indicators of coherence were identified in
  the U.S. lesson? In the Japanese lesson?

23
U.S. Clip 2
24
Japan Clip 3
25
Japan Clip 4
26
Japan Clip 5
27
Compare coding

• Which indicators of coherence were identified in
  the U.S. lesson? In the Japanese lesson?

28
Overall judgment codes

• Degree to which lesson focused on one main
  learning goal
• Judgment of coherence Is there coherent
  development of science ideas, with a strong
  science content storyline?
• 1 no
• 2 poor
• 3 fair
• 4 good
• 5 excellent

29
Part 3 Discussion

• How important is it for the science education
  community to have a shared words-to-images
  language for describing science teaching?
• Do you think its possible for the science
  education research community to develop such
  consensus?
• What will increase the value of such an effort?
• What are the pitfalls?
• What do you think about selecting coherence as
  one of the 5-10 features of science teaching to
  be used in this project?