Measuring the Link between Elementary Teachers and Student Achievement A Presentation of the Dissertation:

1
Measuring the Link between Elementary Teachers
and Student AchievementA Presentation of the
DissertationElementary Teachers and the
Mathematics Achievement of Urban Students

• Alan Spicciati, Ed.D.
• Seattle Pacific University, Class of 2008
• spicciad_at_hsd401.org

2
Research Findings on the Variability of Student
Achievement by Teacher

• The difference between teachers one SD above and
  below the mean is one years worth of achievement
  (Hanushek, 1992)
• Teacher effects are cumulative three years with
  top vs. bottom quintile teachers opens a 54
  percentile gap (Sanders Rivers, 1996)
• Rowan, Correnti, Millers (2002) comprehensive
  study of teacher measurement methodology
  concluded 52-72 of student mathematics variance
  lies between classrooms, with the rest between
  students and between schools.
• A one SD increase in teacher effectiveness is
  equal to a reduction in class size from 25 to 15
  (Nye, Konstantopoulos, Hedges, 2004)

3
Important Findings on Teacher Characteristics

• Experience
• Experience has a curvilinear relationship with
  achievement.
• Achievement rises with experience for between 2
  and 5 years, with on-the-job training, then
  levels off (Ferguson, 1991 Darling-Hammond,
  2000 Rockoff, 2004 Rivkin, Hanushek, Kane,
  2005).

4
Important Findings on Teacher Characteristics

• Advanced Degrees
• Masters degrees are important in mathematics and
  science in secondary (Goldhaber Brewer, 1997
  Wenglinsky, 2000).
• Findings on advanced degrees are split for
  elementary.
• Many studies find that advanced degrees do not
  relate to elementary mathematics
  achievement...(Hanushek, 1986 Rivkin, Hanushek,
  Kain, 2005 Clotfelter, Ladd, Vigdor, 2007).
• However, some reputable studies find a positive,
  significant relationship (Ferguson Ladd, 1996
  Greenwald, Hedges, Laine, 1996 Nye,
  Konstantopolous, Hedges, 2004).

5
Important Findings on Teacher Characteristics

• College Selectivity
• A teachers academic ability, particularly verbal
  ability, is among the most established teacher
  variables in relation to student achievement
  (Hanushek, 1986 Rice, 2003).
• College selectivity, often measured by Barrons
  rankings, is a proxy for academic ability that is
  moderately related to student achievement (Wayne
  Youngs, 2003).

6
Important Findings on Teacher Characteristics

• Mathematics Courses
• Mathematics content knowledge, as measured by
  tests of teachers, relates to achievement
  (Harbison Hanushek, 1992 Hill, Rowan, Ball,
  2005).
• Mathematics courses relate to math achievement in
  secondary (Monk King, 1994).
• However, Hill, Rowan, Ball (2005) found there
  is little empirical evidence examining math
  courses and achievement at the elementary level,
  and their findings were not significant.

7
Definition

• Teacher effectiveness. The present study is
  focused on teachers, as opposed to teaching.
  In this context, teacher effectiveness is
  defined by the mathematics achievement of a
  teachers students, as measured by growth on the
  Measures of Academic Progress (MAP) test,
  compared to expected growth. While teacher
  effectiveness is a term used in the literature,
  this will be a correlational study and will not
  imply effects.

8
Research Questions

• In terms of descriptive statistics, what is the
  distribution of achievement growth at the
  classroom level?
• Is there a significant relationship between
  advanced degrees, experience, college
  selectivity, or total mathematics courses taken
  at the university level and growth in mathematics
  achievement?
• What combinations of the above teacher variables
  best explain the variance in student growth?
• Since poor and minority communities generally
  attract and retain less qualified and experienced
  teachers than other communities, would the
  achievement of diverse classes be significantly
  higher if they had equal or even equitable access
  to teachers with experience and advanced degrees?

9
Participants

• 3,558 students
• 70.7 of all students in grades 3-6
• 84.2 of all students with complete scores,
  excluding self-contained classes
• 156 teachers
• 68.7 of all teachers in grades 3-6
• 89.7 of all eligible teachers
• Required teacher variable data was located for
  all teachers

10
Instrument

• Measures of Academic Progress (MAP)
• Published by Northwest Evaluation Association
  (NWEA)
• Computer adaptive item response theory
• Multiple choice typically 40 items
• Measures the content strands found on the math
  WASL
• Administered fall, winter, and spring
• Reliability and Validity
• Test-retest reliability r .88 to r .93
• Marginal reliability r .94
• Concurrent validity (with state tests) r .79
  to r .89

11
Procedures

• Permission granted by superintendent and SPU
  Institutional Review Board
• Gathered existing data
• MAP scores accessed in raw format from district
  database
• Teacher data accessed from Human Resources
• Degree database contained universities and
  degrees
• Highly Qualified Teacher database contained
  record of course taking
• Samples double checked against actual transcripts

12
Variables

• Independent Variables
• Demographic
• Class Percent Non White (CPNW)
• School Free and Reduced Lunch Percentage (SFRL)
• Class Percent of English Language Learners (CPEL)
• Teacher
• Experience (EXP)
• Experience Dichotomized (EXPDI)
• Degree (DEGR)
• College Selectivity (COLL)
• Number of Mathematics Courses, Content and
  Pedagogy (MC)
• Math Courses Dichotomized (MCDI)
• Dependent Variable
• Class Percent of Expected Growth (CPEG)
• Fall to spring student level MAP growth, divided
  by NWEA expected (normal) growth, aggregated to
  class level

13
Statistical Procedures

• Descriptive Statistics
• Overall
• Disaggregated by quartile level of diversity
• Correlation
• Multiple Regression
• Identification of best model for this dataset
• Regression equation used to estimate results with
  various staffing scenarios

14
Descriptive Statistics
15
Means of variables, disaggregated by class
percent non-white (CPNW) quartile

• Performance
• Least diverse quartile grew most
• Demographics
• Poverty and ELL highly related to diversity
• Teachers
• Low diversity classes taught by more experienced
  teachers
• Other variables have weaker relationships

16
Scatter of Classrooms by Diversity Level and CPEG

• Diversity level only explains 9 of growth
• Large range of growth at every level of diversity
• Many highly diverse classes outperform expected
  growth

17
Performance by diversity quartile, and growth
quartile within diversity quartile

• Explanation
• Each color represents a diversity quartile
• Each bar represents 9 or 10 classrooms, grouped
  by growth, with average CPEG shown
• Interpretation
• Top classrooms in every diversity quartile
  outperform the average non diverse class

18
Intercorrelation of demographic, teacher, and
classroom achievement variables
19
Multivariate linear regression, preliminary/full
model

• Diversity explains 9 of CPEG scores
• Advanced degrees and experience explain an
  additional approximately 9
• Experience does not significantly explain CPEG
  scores beyond advanced degrees

20
Multivariate linear regression, reduced model

• The reduced model includes only diversity and
  advanced degrees
• Advanced degrees explain more than 9 of variance
  in CPEG scores beyond what diversity explains
• The model as a whole explains about 18 of the
  variance in CPEG scores

21
Estimated achievement based on various scenarios
of teacher assignment

• Explanation
• Using Beta weights from the multiple regression
  equation, achievement levels are simulated using
  different allocation methods
• Interpretation
• An equitable approach could close achievement gap
  between Q1 and Q4 from 21 (in the status quo
  model) to 7
• See limitations
• This approach would be more powerful with a
  stronger measure of teacher quality or a
  characteristic that varies more greatly across
  schools

22
Discussion

• Q1 Distribution of Achievement by Classroom
• 1 SD of classroom effectiveness nearly 3 months
  of growth
• Q2 Teacher Characteristics and Math Achievement
  Growth
• Advanced degrees
• Different findings may be attributable to small
  n of colleges, local bargaining context, or
  methodology that cannot link individual teachers
  with their characteristics.
• Experience
• Findings herein consistent with research.
• College selectivity
• Data lacks variability to show results.
• Mathematics courses
• Content knowledge appears to matter, based on
  Hill, Rowan, Ball (2005), but coursework is a
  poor proxy.
• Q3 Combinations of Variables
• No significant interactions.
• Q4 Equal or Equitable Distribution of Teachers
• Simulations of this nature may be needed to
  encourage policy.

23
Implications for Practice

• Knowing and acting on data
• Disaggregating achievement data by classroom
• Using responsible and ethical assessment and HR
  practices
• Engaging in courageous conversations and
  leadership actions
• Teacher distribution and assignment
• Monitor teacher characteristics data to prevent
  neediest schools from having disproportionately
  inexperienced/less qualified teachers
• Referee student assignment to avoid repeated
  exposure to low performing classrooms (Sanders)

24
Limitations

• Methodology
• Gain scores
• Small student n size per teacher
• Multiple regression vs. HLM
• Internal Validity
• Does measuring classrooms measuring teachers?
• Unidentified covariates
• External Validity
• Ability to generalize
• Assumptions that teachers would perform similarly
  in different situations

25
Suggestions for Future Research

1. A multi-state study.
2. A study of teachers that lasted more than one
   year.
3. A study of other forms of mathematics content
   acquisition.
4. A study that includes variables for teachers who
   took a remedial mathematics course or who failed
   a mathematics course.
5. A qualitative study of teachers whose students
   significantly outperform.

26
Measuring the Link between Elementary Teachers
and Student AchievementA Presentation of the
DissertationElementary Teachers and the
Mathematics Achievement of Urban Students

• Alan Spicciati, Ed.D.
• Seattle Pacific University, Class of 2008
• spicciad_at_hsd401.org