Designing Assessments for Mathematical Knowledge for Teaching A Practical Roadmap from Conceptual Fr

1
Designing Assessments forMathematical Knowledge
for TeachingA Practical Roadmap from Conceptual
Framework to Work PlanTexas Regional
CollaborativesProfessional Development Academy
SRI International Austin, TX April 24-25, 2008
2
OverviewandIntroduction
Ma Pa Kettle
3
Purpose of this Workshop

• The need
• The US ED MSP Program requires federally funded
  programs to report observed gains in teacher
  content knowledge
• Measures of teacher content knowledge are not
  necessarily already available
• Overall goals for this workshop
• Discuss in depth the concept of mathematical
  knowledge for teaching (MKT)
• Provide basic knowledge for using, adapting,
  and/or developing assessments for MKT for teachers

4
Workshop Flow

• The construct of MKT
• Gain familiarity with the construct of MKT
• Examine available MKT instruments in the field
• Assessment Design
• Gain familiarity with the Evidence-Centered
  Design approach
• Begin to design a framework for your own
  assessment
• Assessment Development
• Begin to create your own assessment items in line
  with your framework
• Assessment Validation
• Learn basic tools for how to refine and validate
  an assessment
• Plan next steps for using assessments

5
Scope of Purposes for AssessingTeacher MKT

• Research on teaching
• Teacher certification
• Diagnosis for individual teachers where PD is
  necessary
• Discussion points for professional development
• Evaluate knowledge growth that occurs in a PD
  program

Focus is here
6
What We Are Not Emphasizing

• Teachers opinions, attitudes, beliefs
• Teachers enacted practices
• Though similar principles of instrument
  development would apply

7
Important Caution

• Our emphasis is on evaluating programs, not
  individual teachers
• We are not discussing assessments that will have
  consequences for individual teacher evaluation or
  employment decisions

8
Sources of Instruments

• Off-the-shelf
• Adaptations of off-the-shelf
• Newly developed

9
Getting to Know You (N31)
Job Title
10
Getting to Know You (N31)
11
Getting to Know You (N31)
12
Getting to Know You (N31)
13
Getting to Know You

• Name and region
• Setting where you work
• Grade-level focus
• Content focus
• Purpose for attending this workshop

14
What questions do you have that you would like
answered in these two days?

• Can we realistically address reliability/validity
  for small people?
• Is what were doing making a difference to
  student achievement?
• What role does value-added assessment play?
• How do we measure MKT?

15
Tour of the Binder
16
Our ExampleThe Scaling Up SimCalc Project

• The central question
• Can a wide variety of middle school math teachers
  use
• an integration of technology, curriculum, and TPD
• to increase student learning
• of complex and conceptually difficult
  mathematics?
• Two randomized experiments
• 7th Grade 2005-2007
• 8th Grade 2006-2007

17
Worked in 8 Regions in Texas
18
Intervention An Integration of Technology,
Curriculum, and TPD
19
TechnologySimCalc MathWorlds

• Designed to help students make mathematical
  connections
• Dynamic, interactive links between representations

20
Curriculum Teachers Usea 3-Week Replacement Unit

• Proportionality (7th Grade)
• Linear function in the form ykx
• Rate

• Linear Function (8th Grade)
• Linear function in the form ymxb
• Average rate

21
Teacher ProfessionalDevelopment

• Sequence of summer workshops totaling 6 days
• Training and planning opportunities for
• Mathematical content
• Software
• Curriculum materials

22
The Assessments
23
Purposes of theMKT Assessments

• To assess teacher learning in our workshops
• To examine the relationships between student
  learning and teacher MKT

24
Findings Students in SimCalc Classrooms Learn
More Complex Math
7TH GRADE Year 1 (30-item test)
8TH GRADE (36-item test)
Effect size 0.84 1.22 (M2)
Effect size 0.79 1.27 (M2)
25
Teacher MKT Predicts Student Learning Gains in
SimCalc Classrooms