The Judiciary

1

Implementing Merit Pay, Overcoming the Obstacles
Lessons from the Achievement Challenge Pilot
Project
Gary W. Ritter Endowed Chair in Education
Policy Department of Education Reform University
of Arkansas
Presented to New York State Association
of Management Advocates for School Labor
Affairs 30th Annual Summer Conference Saratoga
Springs, NY
2
Presentation Overview

1. Background
2. Teacher Salary as Policy Tool
3. Policy Alternatives
4. LRSD Evaluation

3
Why Focus on Teacher Salaries?

• The research is clear and consistent in
  acknowledging the important role of teachers.
  However, the research is not clear or consistent
  in identifying strategies for recruiting and
  retaining effective teachers.
• Current Single Salary System
• Based on tenure and degree
• Lock-step
• Arguments for single system
• Fair
• Simple

4
Whats Wrong the Status Quo?

• Concerns
• Single salary system does not address teacher
  shortages.
• Geographic area
• Subject area
• Single salary system does not reward student
  outcomes.
• Incentive to leave field (better salary)
• Incentive to transfer schools (better environment)

5
Entry Level Teacher Pay Competitive
New Teacher
New Business Graduate
6
Rewards for Teaching Excellence Decline over Time
7
Rewards for Effective Teachers?
8
Rewards for Effectiveness?
9
Why Consider a Bonus Program?

• Teachers affect student performance, however
• Status Quo
• Single Salary System, based on tenure and degree
• Does not necessarily encourage innovation or
  adopting more effective techniques
• Need
• System to recruit and retain talented teachers
  and to reward high quality instruction
• What alternatives do we have?

10
Three Policy Alternatives to Recruit, Retain, and
Reward Effective Teachers

• Lump Sums
• Does not change incentive structure
• Legislature employed this alternative
• Differential Pay
• Hard-to-staff schools
• Specific subjects
• Legislature employed this alternative
• Merit Pay
• Teacher characteristics (e.g. National Board,
  Prof Dev)
• Teacher behavior
• Student performance

11
Merit Pay Debate

• How does merit pay affect student performance?
• Two types of potential effects
• Composition
• Motivation
• Supporters believe performance improves
• Innovation
• Work harder
• Salary satisfaction
• Opponents believe performance decreases
• Counter-productive competition
• Degraded work environment
• Focus on high-performing students
• What does the evidence say?

12
Merit Pay Literature

• Very few rigorous evaluations
• Many programs are short-lived
• Until recently, data limitations
• Existing evidence indicates
• Teachers often displeased
• According to recent lit review, student
  performance generally improves or stays the same

13
Little Rock School Districts Achievement
Challenge Pilot Project (ACPP)

• Program Goals
• Increase student performance
• Reward effective teachers
• Improve school culture
• 5 elementary schools
• Financial rewards to teachers based on annual
  gains in student performance

14
ACPP Understandable, Non-Competitive for
Teachers, Significant , and Focus on Growth of
Students

• Table 1 Payouts for Wakefield for 2006-07

Employee Type / Position 0-4 Growth 5-9 Growth 10-14 Growth 15 Growth Maximum Payout
Principal 2,500 5,000 7,500 10,000 10,000
Teacher (Grades 4 -5) 50 100 200 400 11,200
Teacher (Grades 1-3) 50 100 200 400 10,000
Teacher (Kindergarten) 50 100 200 400 8,000
Coach 1,250 2,500 3,750 5,000 5,000
Specialist Spec. Ed. 1,000 2,000 3,000 4,000 4,000
Music Teacher 1,000 2,000 3,000 4,000 4,000
Physical Examiner 500 1,000 1,500 2,000 2,000
Aide 250 500 750 1,000 1,000
Secretary Custodian 125 250 375 500 500

15
Student Effects?

• Question
• What is the impact of the ACPP on the math
  performance of students?
• Method
• Student level fixed-effects regression model
• Data provided by the Little Rock School District
• Test scores
• Stanford Achievement Test-9
• Iowa Test of Basic Skills
• Reduces gaming effect
• Demographic data
• Race, Poverty (FRL), Gender, Age

16
Determining Student Effect

• Compare the difference in test scores for ACPP
  students to the difference in test scores for
  comparison students
• Comparison based on math scores of 4th and 5th
  grade students due to data availability
• pre-gains (2002-03 or 2003-04 to 2004-05)
• post-gains (2004-05 to 2005-06)

17
Selecting Comparison Schools
Table 2 Demographic Characteristics of Schools
in 2005-06 ACPP Evaluation
School Name Enrollment, 2005-06 Free/ Reduced Lunch, 2005-06 Black, 2005-06 2-Year Proficient, Math, 2003, 2004 2-Year Proficient, Literacy, 2003, 2004
Meadowcliff 349 90 80 45.8 49.4
Wakefield 445 94 78 47.2 54.0
Baseline 245 96 79 54.9 59.2
Chicot 460 89 80 37.3 44.1
Franklin 363 95 97 25.0 45.3
Treatment 794 93 78 46.5 51.8
Control 1,068 93 85 38.1 48.4
18
Student Effect Results
19
Teacher Survey Hypotheses Tested

• Supporters believe merit pay causes
• Increased effort
• Innovation
• Salary satisfaction
• Opponents believe merit pay causes
• Counter-productive competition
• Divisive work environment
• Focus on high-performing students
• If merit pay is beneficial, then student
  achievement should improve. If merit pay is
  detrimental, then student performance should
  decrease.

58 ACPP teachers 74 comparison teachers
20
Teacher Effect Results
Constructs ACPP ( Agree) Comparison ( Agree) Result
Innovative 88 90 Neutral
Work Harder 86 99 Comp
Satisfaction with Compensation 53 35 ACPP
Collaboration 95 99 Neutral
Positive School Environment 86 56 ACPP
Openness to Challenges 86 66 ACPP
Effective Teacher 90 78 ACPP
21
Policy Implications Conclusions

• ACPP improves student performance
• Student performance increased 3.5 NCE points
  (roughly 6 to 7 percentile points)
• Teachers support the ACPP
• Significantly more satisfied with ACPP than
  single salary system
• Report the program did not lead to
  counterproductive competition
• Repot the school environment is more positive
  with ACPP
• Report ACPP has positive impacts for students
• As policymakers consider ways to recruit, retain,
  and reward effective public school teachers, they
  would do well to consider the results of this
  performance pay plan evaluation.

22
Implications for Program Development

• Why Have Programs Failed in the Past?
• Characteristics of Merit Pay Programs with a
  Chance of Succeeding

23
Example Performance Pay Program (Performance Pay)

• Program Goals
• Increase student performance
• Reward effective teachers
• Make positive influences on school culture
• Ultimately, recruits, retains, and rewards
  effective teachers

24
Performance Pay Possible Design

• Teachers rewarded for
• Classroom gains in student performance as
  measured by Benchmarks and ITBS (30)
• School-wide gains in student performance as
  measured by Benchmarks and ITBS (60)
• Performance evaluation conducted by principal
  (10)
• Ultimately, Performance Pay should increase
  student performance by
• Rewarding effective teachers
• Exerting a positive influence on school culture
• Recruiting and retaining effective teachers

25
Top Ten Obstacles to Merit Pay Programs

• 10. Only benefits teachers of top students
• 9. Based on a secret formula
• 8. Teaching to the test
• 7. Extra testing
• 6. All future raises will be based on test scores

• 5. What about art, PE, and music teachers?
• 4. Discourages collegiality
• 3. Fully-functional human beings
• 2. Teachers do not teach for money
• 1. Rewards arent worth extra work

26
Challenge Number 10

• Complaint
• Only benefits teachers of top students
• Teachers with the easiest students will have an
  unfair advantage in a scheme based on test
  scores.

• Performance Pay
• Teacher rewards should be based on test score
  improvement of all students in a classroom for an
  entire year.
• It may be easier for students at the low end to
  experience improvements!

27
Challenge Number 9

• Complaint
• Based on a secret formula
• It is not at all clear to the teachers how the
  rewards will be decided and whether it is fair.
  If we dont understand what will be rewarded,
  this cannot possibly work.

• Performance Pay
• Teacher rewards should be based on a simple
  calculation of test score improvement.
• The gain measure is then straightforward
  post-score minus the pre-score.

28
Challenge Number 8

• Complaint
• Teaching to the test
• A monetary bonus program will force teachers to
  teach only items on the test and ignore
  everything else.

• Performance Pay
• The benchmark tests are representative of
  curricular frameworks developed by teachers and
  leaders. Teachers will be rewarded to teaching
  skills that educators have decided are important.
  
• Testing is not the only component of Performance
  Pay

29
Challenge Number 7

• Complaint
• Extra Testing
• Students will now have to spend more time testing
  and even less time in the classroom being
  instructed by effective teachers.

• Performance Pay
• Teacher rewards can be based on student scores
  from tests that are already administered.
• No new testing would be required.

30
Challenge Number 6

• Complaint
• All future raises will be based on test scores
• District leaders will now shift all new funds for
  salary increases into the bonus program and
  eliminate other raises!

• Performance Pay
• A successful program might focus only on bonuses
  (additions to standard salary).
• Performance Pay would likely have no impact on
  future teacher salary increases.

31
Challenge Number 5

• Complaint
• What about art, PE, and music teachers?
• Bonus programs only reward teachers of core
  subjects and ignore all other important subjects.
• Other subjects will then be marginalized in the
  schools.

• Performance Pay
• Teacher rewards based on school wide student
  growth and principal evaluations.
• Other subjects contribute to school wide growth
  and the job of the principal, whose rating is
  incorporated, is to ensure that all subjects be
  given a full treatment.

32
Challenge Number 4

• Complaint
• Discourages collegiality
• Teachers will no longer want to work together
  because they will begin to view their colleagues
  as their competitors in a race for bonus money.

• Performance Pay
• Teacher rewards will not be treated as a
  zero-sum game.
• Teacher rewards are not decreased by rewards
  given to peers. Rewards based on student
  improvement and all teachers may be rewarded.
• Survey data reveals additional collaboration in
  schools with such programs.

33
Challenge Number 3

• Complaint
• Fully functional human beings
• Teachers will be rewarded for producing students
  who can successfully fill in bubble sheets
  instead of producing fully functioning human
  beings.

• Performance Pay
• Teacher rewards based on exams that educators
  believe are important indicators of readiness for
  self-sufficiency.
• The ability to be fully functioning is related to
  learning key skills that the tests measure.

34
Challenge Number 2

• Complaint
• Teachers do not teach for money.
• Monetary rewards are not important to teachers.
  Teachers do not enter this profession to become
  rich.

• Performance Pay
• Teachers are likely similar to most people they
  have many preferences, they prefer more money to
  less, and they prefer to be rewarded for good
  work.
• Teachers, like other workers, do seek salary
  increases.

35
Challenge Number 1

• Complaint
• Rewards arent worth it.
• A reward of one or two thousand dollars cannot
  motivate great changes on a daily basis.
• Many past similar teacher award programs have
  been viewed as unsuccessful, partially because of
  low levels of rewards.

• Performance Pay
• Depending on the model, effective teachers can a
  earn maximum reward of
• 10,000

36
Contact InformationGary Ritter, Associate
ProfessorOffice for Education PolicyUniversity
of Arkansashttp//www.uark.edu/ua/oepEmail
oep_at_uark.eduPhone [phone]