Teacher Quality, Quality Teaching, and Student Outcomes: Measuring the Relationships - PowerPoint PPT Presentation

About This Presentation
Title:

Teacher Quality, Quality Teaching, and Student Outcomes: Measuring the Relationships

Description:

Mathematical Knowledge for Teaching MKT Items MKT Findings History of Mathematical Quality of Instruction (MQI) MQI: Sample Domains and Codes Initial study: ... – PowerPoint PPT presentation

Number of Views:92
Avg rating:3.0/5.0
Slides: 34
Provided by: Heather302
Learn more at: https://cadrek12.org
Category:

less

Transcript and Presenter's Notes

Title: Teacher Quality, Quality Teaching, and Student Outcomes: Measuring the Relationships


1
Teacher Quality, Quality Teaching, and Student
Outcomes Measuring the Relationships
  • Heather C. Hill
  • Deborah Ball, Hyman Bass, MerrieBlunk, Katie
    Brach,
  • CharalambosCharalambous, Carolyn Dean, Séan
    Delaney, Imani Masters Goffney, Jennifer Lewis,
    Geoffrey Phelps, Laurie Sleep, Mark Thames,
    Deborah Zopf

2
Measuring teachers and teaching
  • Traditionally done at entry to profession (e.g.,
    PRAXIS) and later informally by principals
  • Increasing push to measure teachers and teaching
    for specific purposes
  • Paying bonuses to high-performing teachers
  • Letting go of under-performing (pre-tenure)
    teachers
  • Identifying specific teachers for professional
    development
  • Identifying instructional leaders, coaches, etc.

3
Methods for identification
  • Value-added scores
  • Average of teachers students performance this
    year differenced from same group of students
    performance last year
  • In a super-fancy statistical model
  • Typically used for pay-for-performance schemes
  • Problems
  • Self-report / teacher-initiated
  • Typically used for leadership positions,
    professional dev.
  • However, poor correlation with mathematical
    knowledge
  • R 0.25

4
Identification Alternative Methods
  • Teacher characteristics
  • NCLBs definition of highly qualified
  • More direct measures
  • Educational production function literature
  • Direct measures of instruction
  • CLASS (UVA)general pedagogy
  • Danielson, Saphier, TFAditto
  • But what about mathematics-specific practices?

5
Purpose of talk
  • To discuss two related efforts at measuring
    mathematics teachers and mathematics instruction
  • To highlight the potential uses of these
    instruments
  • Research
  • Policy?

6
Begin With Practice
  • Clips from two lessons on the same content
    subtracting integers
  • What do you notice about the instruction in each
    mathematics classroom?
  • How would you develop a rubric for capturing
    differences in the instruction?
  • What kind of knowledge would a teacher need to
    deliver this instruction? How would you measure
    that knowledge?

7
Bianca
  • Teaching material for the first time (Connected
    Mathematics)
  • Began day by solving 5-7 with chips
  • Red chips are a negative unit blue chips are
    positive
  • Now moved to 5 (-7)
  • Set up problem, asked students to used chips
  • Given student work time

8
Question
  • What seems mathematically salient about this
    instruction?
  • What mathematical knowledge is needed to support
    this instruction?

9
Mercedes
  • Early in teaching career
  • Also working on integer subtraction with chips
    from CMP
  • Mercedes started this lesson previous day,
    returns to it again

10
Find the missing part for this chip problem.
What would be a number sentence for this problem?
Start With Rule End With
Add 5
Subtract 3
11
Questions
  • What seems salient about this instruction?
  • What mathematical knowledge is needed to support
    this instruction?

12
What is the same about the instruction?
  • Both teachers can correctly solve the problems
    with chips
  • Both teachers have well-controlled classrooms
  • Both teachers ask students to think about problem
    and try to solve it for themselves

13
What is different?
  • Mathematical knowledge
  • Instruction

14
Observing practice
  • Led to the genesis of mathematical knowledge for
    teaching
  • Led to mathematical quality of instruction

15
Mathematical Knowledge for Teaching
Source Ball, Thames Phelps, JTE 2008
16
MKT Items
  • 2001-2008 created an item bank of for K-8
    mathematics in specific areas (see
    www.sitemaker.umich.edu/lmt) (Thanks NSF)
  • About 300 items
  • Items mainly capture subject matter knowledge
    side of the egg
  • Provide items to field to measure professional
    growth of teachers
  • NOT for hiring, merit pay, etc.

17
MKT Findings
  • Cognitive validation, face validity, content
    validity
  • Have successfully shown growth as a result of
    profl development
  • Connections to student achievement - SII
  • Questionnaire consisting of 30 items (scale
    reliability .88)
  • Model Student Terra Nova gains predicted by
  • Student descriptors (family SES, absence rate)
  • Teacher characteristics (math methods/content,
    content knowledge)
  • Teacher MKT significant
  • Small effect (lt 1/10 standard deviation) 2 - 3
    weeks of instruction
  • But student SES is also about the same size
    effect on achievement
  • (Hill, Rowan, and Ball, AERJ, 2005)
  • Whats connection to mathematical quality of
    instruction??

18
History of Mathematical Quality of Instruction
(MQI)
  • Originally designed to validate our mathematical
    knowledge for teaching (MKT) assessments
  • Initial focus How is teachers mathematical
    knowledge visible in classroom instruction?
  • Transitioning to What constitutes quality in
    mathematics instruction?
  • Disciplinary focus
  • Two-year initial development cycle (2003-05)
  • Two versions since then

19
MQI Sample Domains and Codes
  • Richness of the mathematics
  • e.g., Presence of multiple (linked)
    representations, explanation, justification,
    multiple solution methods
  • Mathematical errors or imprecisions
  • e.g., Computational, misstatement of mathematical
    ideas, lack of clarity
  • Responding to students
  • e.g., Able to understand unusual
    student-generated solution methods noting and
    building upon students mathematical
    contributions
  • Cognitive level of student work
  • Mode of instruction

20
Initial study Elementary validation
  • Questions
  • Do higher MKT scores correspond with
    higher-quality mathematics in instruction?
  • NOT about reform vs. traditional instruction
  • Instead, interested in the mathematics that
    appears

21
Method
  • 10 K-6 teachers took our MKT survey
  • Videotaped 9 lessons per teacher
  • 3 lessons each in May, October, May
  • Associated post-lesson interviews, clinical
    interviews, general interviews

22
Elementary validation study
  • Coded tapes blind to teacher MKT score
  • Coded at each code
  • Every 5 minutes
  • Two coders per tape
  • Also generated an overall code for each lesson
    low, medium, high knowledge use in teaching
  • Also ranked teachers prior to uncovering MKT
    scores

23
Projected Versus Actual Rankings of Teachers
Projected ranking of teachers Actual
ranking of teachers (using MKT scores)
Correlation of .79 (p lt .01)
Hill, H.C. et al., (2008) Cognition and
Instruction
24
Correlations of Video CodeConstructs to Teacher
Survey Scores
Construct (Scale) Correlation to MKT scores
Responds to students 0.65
Errors total -0.83
Richness of mathematics 0.53
significant at the .05 level
25
Validation Study II Middle School
  • Recruited 4 schools by value-added scores
  • High (2), Medium, Low
  • Recruited every math teacher in the school
  • All but two participated for a total of 24
  • Data collection
  • Student scores (value-added)
  • Teacher MKT/survey
  • Interviews
  • Six classroom observations
  • Four required to generalize MQI used 6 to be sure

26
Validation study II Coding
  • Revised instrument contained many of same
    constructs
  • Rich mathematics
  • Errors
  • Responding to students
  • Lesson-based guess at MKT for each lesson
    (averaged)
  • Overall MQI for each lesson (averaged to teacher)
  • G-study reliability 0.90

27
Validation Study IIValue-added scores
  • All district middle school teachers (n222) used
    model with random teacher effects, no school
    effects
  • Thus teachers are normed vis-à-vis performance of
    the average student in the district
  • Scores analogous to ranks
  • Ran additional models similar results
  • Our study teachers value-added scores extracted
    from this larger dataset

28
Results
MKT MQI Lesson-based MKT Value-added score
MKT 1.0 0.53 0.72 0.41
MQI 1.0 0.85 0.45
Lesson-based MKT 1.0 0.66
Value added score 1.0
  • Significant at plt.05
  • Significant at plt.01

Source Hill, H.C., Umland, K. Kapitula, L. (in
progress) Validating Value-Added Scores A
Comparison with Characteristics of Instruction.
Harvard GSE Authors.
29
Additional Value-Added Notes
  • Value-added and average of
  • Connecting classroom work to math 0.23
  • Student cognitive demand 0.20
  • Errors and mathematical imprecision -0.70
  • Richness 0.37
  • As you add covariates to the model, most
    associations decrease
  • Probably result of nesting of teachers within
    schools
  • Our results show a very large amount of error
    in value-added scores

30
Lesson-based MKT vs. VAM score
31
Proposed Uses of Instrument
  • Research
  • Determine which factors associate with student
    outcomes
  • Correlate with other instruments (PRAXIS,
    Danielson)
  • Instrument included as part of the National
    Center for Teacher Effectiveness, Math Solutions
    DRK-12 and Gates value-added studies (3)
  • Practice??
  • Pre-tenure reviews, rewards
  • Putting best teachers in front of most at-risk
    kids
  • Self or peer observation, professional development

32
Problems
  • Instrument still under construction and not
    finalized
  • G-study with master coders indicates we could
    agree more among ourselves
  • Training only done twice, with excellent/needs
    work results
  • Even with strong correlations, significant amount
    of error
  • Standards required for any non-research use are
    high

KEY Not yet a teacher evaluation tool
33
Next
  • Constructing grade 4-5 student assessment to go
    with MKT items
  • Keep an eye on use and its complications

Questions?
Write a Comment
User Comments (0)
About PowerShow.com