Public Forum on New York States NCLB Growth Model Proposal

1
Public Forum on New York States NCLB Growth
Model Proposal

• David Abrams
• Assistant Commissioner for Standards, Assessment,
  and Reporting
• Ira Schwartz
• Coordinator, Accountability, Policy, and
  Administration
• Fall 2008

2
Purpose of No Child Left Behind

• to ensure that all children have a fair, equal,
  and significant opportunity to obtain a
  high-quality education and reach, at a minimum,
  proficiency on challenging state academic
  achievement standards and state academic
  assessments.

3
Chapter 57 of the Laws of 2007 Growth Model

• By the start of the 2008-2009 school year, the
  Regents shall establish, using existing state
  assessments, an interim, modified accountability
  system for schools and districts that is based on
  a growth model, subject to approval of the United
  States department of education where required
  under federal law.

4
Next Generation Accountability System Design

• Key Questions
• How do we design accountability models that move
  students from low performance to proficient as
  well as from proficient to distinction?
• How can we ensure that improved scores represent
  improved learning?
• How do we take data and turn it into actionable
  information that improves teaching and learning?
• How do we move from beating the odds to changing
  the odds?

5
Accountability Status vs. Growth

• Status Models take a snapshot of a subgroups or
  schools level of student proficiency at one
  point in time and often compare that proficiency
  level with an established target.
• Growth Models measure progress by tracking the
  achievement scores of the same students from one
  year to the next to determine student progress.

6
Why Growth?
Types of Performance
Status/Growth Combinations
Status
Change
High Status
Achieve-ment
Improve-ment
Status
High/Low
High/High
Effective-ness
Growth
Acceleration
Low/Low
Low/High
Low Status
Low Growth
High Growth
7
Betebenner, Jan. 2008, for RI project
8
Two Types of Growth Targets

• Policy driven targets start with a policy goal
  (what should be) and then establish the targets
  for performance that are necessary to achieve
  this goal.
• Data driven targets start with historical
  performance (what has been) and use that as a
  basis to project what should be expected of the
  units to be measured.

9
New York States Proposal

• Use growth in two different ways
• To make more refined AYP determinations (must be
  approved by USED)
• To supplement AYP and make a more comprehensive
  system, attending in particular to growth of
  students who are proficient or higher (not
  necessary to be approved by USED)

10
Constraints

• Using growth in AYP is highly constrained
• Focused on reaching proficiency and reducing the
  achievement gap
• Specifics dictated by USED
• Proposal must be submitted by Oct. 15 to USED if
  approved, would apply to 2008-09 data
• Using growth outside of AYP is less constrained,
  but there is less agreement on approaches
• NYSED will work with partners to create this
  growth proposal
• Regents would like proposal by end of this school
  year

11
New York State Local Initiatives

• A number of NYS districts have developed local
  growth and value-added models. The two most
  prominent are
• NYCs Progress Report initiative.
• Capital Region BOCES initiative.
• These initiatives are neither endorsed by SED nor
  require SEDs endorsement.
• These initiatives are not constrained by USEDs
  growth model guidelines and were not designed for
  use in making AYP decisions.
• These models can inform the development of State
  growth and/or value-added models for which USEDs
  permission is not required.

12
SEDs Interim Growth Model Design Principles

• Interim Growth Model shall be implemented in
  08-09 school year (with Regents and USED
  approval).
• Model shall meet core principles of Spellings
  11/21/05 correspondence.
• Model shall be based upon NYs current State
  assessment program shall not require the
  implementation of new assessments.
• Model shall utilize such data as is currently
  collected through State data collection processes
  and shall not require the collection of new data
  elements.
• Models purpose shall be to make more refined
  determinations of student progress, identify with
  greater precision high performing schools and
  districts, and support greater differentiation in
  support and services to schools and districts in
  need of improvement.

13
SEDs Interim Growth Model Design Principles

• The model shall be based upon measuring whether
  students are proficient or on track towards
  proficiency within a prescribed time period.
• Model shall use an open architecture. All
  calculations should be transparent.
• The interim growth model shall be a stage in a
  process leading, by 2010-11, to the development
  of an enhanced system that includes a value-added
  model.
• The NCLB model should be combined with a State
  model that includes consequences and/or
  incentives for promoting growth for all students,
  while placing no school or district at risk of
  failing to make AYP, if it would make AYP under
  the current status model.

14
USED Seven Core Principles

• All students proficient or on track to
  proficiency by 2014 set annual goals to close
  subgroup gaps.
• Expectations for annual achievement based on
  meeting grade level proficiency, not based on
  demographic characteristics or school
  characteristics.
• Produce separate decisions for math and
  ELA/reading.
• Include students, subgroups, schools, and
  districts in accountability.

15
USED Core Principles cont.

• Assessment data annual, 3-8 high school,
  operational for more than one year (i.e., at
  least two years of data), produce comparable
  results grade-to-grade and year-to-year approved
  in Peer Review.
• Track student progress (longitudinal).
• Include participation rates and additional
  academic indicator for accountability.

16
USED Peer Review Additional Specifications

• Fully approved assessment system
• Uniform minimum-n for all groups
• No confidence intervals for growth
• Very limited recalculation of student growth
  target
• Cannot use with multiple other non-Status
  approaches, such as Safe Harbor and Index
• Apply growth to all students for reporting and
  accountability (preference)
• Report student growth results (preference)
• State has vertical scale (preference?)

17
SED Draft Proposal

• For grades 3-8, utilize a proficiency plus
  growth model for grades 3-8 similar to North
  Carolinas approved model.
• For high school, utilize a value tables model
  similar to Delawares approved grades 3-8 model.
• Include an enhanced middle level and high school
  component in the proposal.
• Build a growth for all State component that
  sets growth targets for all students, including
  those who are already proficient.

18
NCLB Growth Model General Approach

• If a student scores proficient or above (Level
  3/4) in the current year, include that students
  results in the Performance Index as is done under
  the present status model.
• Use growth to check whether students who did not
  yet score proficient (Level 2) have grown enough
  that it is likely they will become proficient
  within a designated amount of time.
• For purposes of calculating the Performance
  Index, give schools and districts full credit
  for any student who either scores proficient or
  above or who is deemed to be on track for
  proficiency.

19
On-Track Growth to ProficiencyExample
Prof. 8
Prof. 7
Prof. 6
Observed growth Gr. 3-4 projected to Gr. 8
Proficient
Prof. 5
Proficient or
Prof. 4
On track to be proficient
Prof. 3
3 4 5
6 7
8
20
3-8 Growth Model Simplified Example

• Level 3 Scale Score 650.
• Billy scores a 614 in Grade 3 ELA.
• Billy is 36 points below proficiency (650- 614).
• Billy has four years to become proficient.
• Billy must close the gap by ¼ (9 points) in Grade
  4.
• Billys proficiency target in Grade 4 is 623 (614
  9).
• Billy scores 635 in Grade 4.
• Billy now has three years to become proficient.
• Billy must close the gap by 1/3 (5 points) in
  Grade 5.
• Billys proficiency target in Grade 5 is 640 (635
  5).

21
Growth Model Middle School Extension

• Students in middle school would be evaluated on
  whether they made sufficient growth to become
  proficient by the designated high school Regents
  examination.
• The designated high school target is proficient
  on the Regents Examination in Integrated Algebra
  and proficient on the Regents Comprehensive
  Examination in English.
• Students in middle school would have until the
  target assessment to be projected proficient the
  number of years permitted would be based upon the
  grade the student entered middle school.
• This middle school extension will only apply to
  schools (and their subgroups), not to district
  AYP decisions in instances where students
  transfer among schools within a district.

22
Middle Level Extension Simplified Example

• Level 3 Math Regents Exam is equated to scale
  score of 663.
• Billy scores a 623 in Grade 5 Math.
• Billy enrolls in a new middle school in Grade 6.
• Billy is 40 points below proficiency (663-623).
• Billy has four years to become proficient.
• Billy must close the gap by 1/4 (10 points) in
  Grade 6.
• Billys proficiency target in Grade 6 is 633 (623
  10).
• If Billy remained in his original school in Grade
  6, then his proficiency target would have been
  637.
• (650-623 27/2 years 13.5 623 637.)
• Actual equated score not yet determined
• Represents Grade 7 Level 3 Scale Score

23
3-8 Growth Model Implications for Schools
24
Growth Model High School Extension

• Students who enter high school having scored
  Level 1 or at low-Level 2 on the Grade 8 ELA or
  Mathematics tests are considered on track towards
  proficiency if they score between 55-64 on the
  designated Regents examinations prior to Grade
  12.
• Schools have five years for certain English
  Language Learners, certain students with
  disabilities, and students who enter high school
  far below standards to demonstrate proficiency in
  English language arts and mathematics.
• Value table is interim model to be used for
  cohorts prior to the 2008-2009 school year cohort
  (i.e. next three years).

25
Growth Model High School Values Table
26
Timeline for NCLB Growth Models

• July/September 2008 Proposed model submitted
  to Board of Regents for Review.
• September/October, 2008 Discussion with the
  Field of the Model.
• October, 2008 Submission to USED.
• Fall 2008 Approval of model by USED.
• September 2009 Use of model to make AYP
  decisions based on 2008-09 school year data,
  subject to availability of resources.

27
Building a Growth for All Model

• Regents have directed SED to provide
  recommendations for how to hold schools and
  districts accountable for growth of students
  beyond proficiency as part of the process of
  moving towards creation of a value-added
  accountability model.
• This growth for all model can be separate from
  NYs NCLB accountability system and need not be
  constrained by NCLB growth model rules.
• The Regents will need to decide what rewards
  and/or consequences should be based upon a
  growth for all model.

28
Building a Growth for All Model

• One possibility would be to modify the current
  process for designation of High Performing and
  Rapidly Improving schools to include a growth
  for all component other possibilities include
  rewards, regulatory relief, and differentiated
  consequences.

29
Approaches to Growth for All Models

• Student growth in terms of what other reference
  schools or reference groups have achieved (e.g.
  peer schools, low-income Hispanic students).
• Growth of students compared to other students who
  started with similar growth histories.
• Student growth in relation to statistical
  expectations for what the student would have
  learned with a typical teacher/school.

30
Define Value-added

• Increase over previous score or performance
• Increase over what was expected
• Attribution of performance changes
  (increases/decreases) to agents/conditions

31
What do we value?

• A Years Worth of Growth
• More than A Years Worth of Growth
• Not Going Backwards
• Relative Growth
• Absolute Growth
• Growth to Proficiency
• Growth to a Point Beyond Proficiency

32
What happens next?

• Questions and Answers
• Small group discussions and completion of
  surveys
• Summary and next steps

33
More Information

• To submit questions or requests for more
  information, please e-mail
• growthmodels_at_mail.nysed.gov