Student Outcomes and Principal Evaluation: Key Questions for PEAC Principal Evaluation Subcommittee - PowerPoint PPT Presentation

Loading...

PPT – Student Outcomes and Principal Evaluation: Key Questions for PEAC Principal Evaluation Subcommittee PowerPoint presentation | free to view - id: 7f5c3e-ZmJiY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Student Outcomes and Principal Evaluation: Key Questions for PEAC Principal Evaluation Subcommittee

Description:

Student Outcomes and Principal Evaluation: Key Questions for PEAC Principal Evaluation Subcommittee March 8, 2011 – PowerPoint PPT presentation

Number of Views:44
Avg rating:3.0/5.0
Slides: 21
Provided by: ilu95
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Student Outcomes and Principal Evaluation: Key Questions for PEAC Principal Evaluation Subcommittee


1
Student Outcomes and Principal Evaluation Key
Questions for PEAC Principal Evaluation
Subcommittee
March 8, 2011
2
Overview of the Webinar
  • Review of guiding questions for sub-committee
    consideration
  • Introduction and review of value-added measures
    and update on value-added models being created in
    CPS
  • Discussion of guiding questions

3
Overview of the Webinar
  • Review of guiding questions for sub-committee
    consideration
  • Introduction and review of value-added measures
    and update on value-added models being created in
    CPS
  • Discussion of guiding questions

4
Guiding questions on student outcomes
  • What measures should be used in evaluating
    principals?
  • What is the right balance between value-added
    growth measures and attainment measures?
  • How, if at all, should be adjust our judgments
    based on a schools demographics and other
    characteristics, like student mobility?
  • How many years of data should be used for any
    single years rating on student growth?
  • What processes and parameters should guide local
    flexibility and adaptation to the state system
    over time?
  • For each of these categories, we identify
    specific questions (noted in bold) and
    considerations based on research and our
    experience (noted in italics).

5
Measures of Student Outcomes K-8
  • Should we use ISAT data?
  • Better matched to principal evaluation than
    teacher evaluation
  • Larger pool of students in growth analyses allows
    for less variability in direction of results
  • Clearer attribution of students to principal
    (with clear mobility parameters)
  • Serves as one important element of student
    outcomes piece, but helpful if balanced with
    non-test outcomes (in high school) or other
    assessment data (as long as it is consistent
    across the LEA)
  • Important to use multiple years of information to
    establish trend
  • Can be used to measure attainment (e.g., of
    kids meeting proficiency), gain/growth (e.g.,
    increase in of kids meeting proficiency), and
    value-added
  • Should we use interim assessments?
  • Technically sound but some cautions
  • More reliable than summative tests if computer
    adaptive
  • Assessments may not cover all content
  • Students may not take interim assessments
    seriously
  • Such assessments not meant for use as
    accountability tools
  • From 2014-15, the PARCC assessments should
    provide an integrated solution to interim and
    summative assessments

6
Measures of Student Outcomes K-8 (continued)
  • Should we use school-level student outcome goals
    set by principals and their managers?
  • Common practice, but depends on rigor of
    principal manager expectations
  • What other measures of student growth, beyond
    state tests, should we consider?
  • Measures of student aspirations toward college in
    middle school grades
  • Student attendance

7
Measures of Student Outcomes High School
Considerations
  • Should we use PSAE data?
  • Can be used to assess subjects beyond reading and
    math (i.e., writing, science)
  • Can be used as an attainment measure ( of
    students reaching proficiency) and as a growth
    (increase in of students reaching proficiency)
  • Substantial technical issues in converting these
    data to value-added estimates
  • Gap between 8th grade ISAT and 11th grade PSAE,
    with data distortion from dropouts and retained
    students
  • Anticipate improved ability to make value-add
    estimates using PARCC assessments in 2014-15 and
    onward
  • What other measures of student growth, beyond
    state tests, should we consider?
  • High school student growth measures should expand
    beyond state tests to include on track to
    college measures
  • Student attendance
  • Grade to grade progression
  • Credit accumulation (potentially including
    quality of credits)
  • Cohort graduation rates, and quality of diploma
    earned (if data exists)
  • Note These measures can be turned into value
    added metrics, by looking at predicted values
    versus actual values at the school level

8
Balancing attainment, growth, and value-add
  • How should we weight attainment, growth and
    value-add within an overall rating?
  • Focusing on more on movement measures (i.e.,
    gain/growth, value-add)
  • Provides a better picture of the impact of the
    principal
  • Creates stronger incentives for principals to
    work in lower performing schools
  • Pushes schools with higher performing incoming
    students to keep advancing their performance
    (past proficiency to college-ready)
  • Values all students by assessing progress from
    their starting points
  • Requires districts to look at same-student
    comparisons rather than cohort to cohort
    comparisons whenever possible
  • Where possible, use multiple growth measures
  • Relative weight on attainment (or on maintenance
    of growth) might increase as performance level of
    school increases
  • Should we treat low-performing schools and
    high-performing schools differently or the same?
  • There is a ceiling to growth on proficiency,
    suggesting two changes for high-performing
    schools
  • Give schools gain/growth points if they exceed a
    proficiency ceiling (e.g. Chicago approach)
  • Tie a portion of the gain/growth goal to their
    success in increasing the percent of students
    meeting the advanced category on current
    assessments

9
Balancing attainment, growth, and value-add An
illustration
Shift to growing the percentage of students
reaching advanced
Emphasis on measures of growth
Reward principals for maintaining high levels of
achievement
10
DCPS Principal Evaluation Components
Evaluation Component allocated Total
Professional Competencies Professional Competencies 30
Leadership Framework Assessments 30 30
Student Outcomes Student Outcomes 50
Value-Added Measure 20 50
School Specific Goals 10 50
DC CAS Gains Goals 20 50
Other Other 20
Special Education Compliance 10 20
Teacher Retention 5 20
Family Engagement 5 20
11
New York City Principal Evaluation Components
Evaluation Component allocated
Schools Graded Progress Components of the Grade Student growth measures make up 60 Absolute performance 25 School Climate 15 32
School Specific Goals 31
Compliance with District Mandates 15
School Quality Review 22
  • 40-50 of the New York Evaluation is made up of
    Student Outcome data
  • 26 of a schools graded progress is focused on
    student outcomes
  • 14-24 of the School Specific Goals are focused
    on student outcomes

In New York City, a School Quality Review is a
two- or three-day visit by experienced educators
to a school. The visit typically includes
classroom observations, conversations with school
leaders and stakeholders, and examinations of
student work. New York City has developed a
rubric to guide the visits and to determine how
well organized a school is to educate its
students.
12
Chicago Performance Calculators for Principals
Elementary School High School
Attainment 43 36
Gain/Growth 43 64
Value-Added 14 0

Attainment (Status) ISAT targets in reading, math, science, composite and highest grade Attendance target Targets for ACT average One-year dropout and freshmen on track Attendance PSAE Reading/Math/Science
Gain/Growth (Trend) Growth in ISAT in reading, math, science, composite and highest grade Growth in attendance Growth in ACT average One-year dropout and freshmen on track Attendance PSAE Reading/Math/Science AP enrollment and success Reading and Math scores from Explore/Plan/ACT sequence
Value-Added (Growth) Reading and math (ISAT)
13
Adjusting for student characteristics
  • Should we include controls in the value-added
    growth models to account for student
    characteristics?
  • Increases the accuracy of value-added estimates
  • Controls can be changed from year to year to
    alter the approach to a given population (e.g.,
    special education, English language proficiency,
    homelessness)  
  • There may be some value in excluding some
    controls at the sake of maximal accuracy of
    estimates in order to signal heightened
    responsibility for schools to accelerate
    achievement for low income students of color.
  • Should we give extra weight for improving results
    for students who start out further behind?
  • Set targets that expect faster growth for lower
    performing students in the district/state
  • How should we address the question of student
    mobility?
  • VARC and others use methods that assign portions
    of value-added growth to a school based on the
    percentage of the school year a student has been
    enrolled at the school.

14
Years of data used for judgments of principals
  • How many years of data should be used for any
    single years rating on student growth?
  • Given the variation in single-year results,
    evaluate student outcomes based on multi-year
    trends
  • Note Value-added estimates are more reliable at
    the school-level than at the classroom level,
    since higher student numbers reduce the impact of
    year-to-year fluctuations. BUT, we want to create
    incentives for long-term improvement, not quick
    fixes.
  • Provide additional time or use more years of data
    for early tenure principals
  • Plan for the availability of sufficient data
    before any significant consequences (e.g.
    ensuring most recent test data is available
    before making spring retention decisions)

15
Processes for adaptation
  • What guidelines do we put in place for all
    districts to follow if they want to design their
    own systems?
  • The balance of growth and attainment should be
    fixed.
  • Measuring success in other academic subjects
    depends on the presence of reliable local
    assessments.
  • The technical capability to develop and implement
    value-added models is not present in most
    districts.
  • What should be the ongoing process for evaluating
    the system and adapting it?
  • Among other things, the state will need to adjust
    its test measures when the PARCC assessments are
    rolled out in 2014-15

16
Overview of the Webinar
  • Review of guiding questions for sub-committee
    consideration
  • Introduction and review of value-added measures
    and update on value-added models being created in
    CPS
  • Discussion of guiding questions

17
Common Approaches to Measuring Student Success
Our overall goal is to measure the performance of
a principal based on student performance. How is
this accomplished?
Source VARC (http//varc.wceruw.org/tutorials/Oak
/index.htm)
18
Understanding Value-Added Measures
  • Stephen Ponisciak
  • Value-Added Research Center
  • School of Education, University of
    Wisconsin-Madison

19
Overview of the Webinar
  • Review of guiding questions for sub-committee
    consideration
  • Introduction and review of value-added measures
    and update on value-added models being created in
    CPS
  • Discussion of guiding questions

20
Guiding questions on student outcomes
  • What measures should be used in evaluating
    principals?
  • What is the right balance between value-added
    growth measures and attainment measures?
  • How, if at all, should be adjust our judgments
    based on a schools demographics and other
    characteristics, like student mobility?
  • How many years of data should be used for any
    single years rating on student growth?
  • What processes and parameters should guide local
    flexibility and adaptation to the state system
    over time?
About PowerShow.com