Student Longitudinal Growth Project

1
Student Longitudinal Growth Project

• Jonathan Wiens
• Office of Assessment and Information Services
• Oregon Department of Education

2
Model Development

• 1/08 to 8/08 ODE researches growth models, and
• tests them with historic test data
• 3/08-9/08 ODE shares models with internal and
• external advisory committees
• 9/08-11/08 ODE, stakeholders, and technical
• consultant refine details of leading
• candidates, with stakeholder
  input
• 11/08-12/08 ODE, advisors recommend the
  Student Centered model
• 12/08- 4/09 ODE presents Student Centered
• Growth model for wider review.

3
Growth Models

• A Growth Model is a method of measuring student
  learning over time.
• Growth models can provide a more dynamic view of
  school effectiveness.
• Growth models can allow schools receive credit
  for
• closing the Achievement Gap
• accelerating student learning
• moving students closer to standard

4
Individual Student Growth

• The most detailed view of growth occurs at the
• student level.
• Which years are a success for the student and for
  
• the school?
• - The Gap is the difference between the
  student score and the cut score.

5
The Student Centered Model

• Students below standard in the previous year are
  given a target score for the current year.
• Students whose score is at or above their target
  meet growth.
• Targets are based on the students prior year
  test score.
• Students are expected to grow toward meeting
  standard.
• Lower performing students must exhibit higher
  growth.
• - does not apply to 10th grade.

6
Sample Reading Growth Targets Growth
targets will require the gap between a
students score and benchmark to decrease over
time, generally by about 40 to 50 each year.
7
Required Gap closures

• The gap closures below are based on growth in the
  meets scores over one- and three- year periods.

8
Median Reading Growth Actual Data
9
Sample Trajectories

• The next charts show the trajectories of students
  who exactly meet their growth targets each year.
• The charts include the option for growth toward
  a 236 for those meeting or exceeding (these are
  still draft and are not used in accountability).
• Four examples are given one student each at the
  Very Low, Low, Meets, and Exceeds.

10
(No Transcript)
11
(No Transcript)
12
07-08 Statewide Results
13
(No Transcript)
14
Percent Meeting Growth Based on Prior Year Score
15
Reading Longitudinal Student Growth Report
Mock-Up
16
School Centered Model

• Examines the effect of a students prior year
  test score on the odds a student will meet in the
  current year.
• Suggested by American Institutes of Research
  (AIR), which has worked on growth with two other
  states.

17
Percent Meeting in 4th Grade by 3rd Grade Score
Statewide Results 2006-07 to 2007-08
18
Range of School Effects
This chart shows the models highest and lowest
performing schools at 4th grade in 2007-08.
Notice the wide range in school effects.
19
Probability Spreadsheet

• The spreadsheet contains district level data.
  ODE plans to release this data before the end of
  the school year.

20
(No Transcript)
21
Report Card School Rating Formula
22
Timeline

• Sept 24 Preview of Detail sheets
• Oct 1 Preview of Summary sheets
• Oct 15 2nd preview
• Oct 29 3rd preview
• Nov 5 Final district preview
• Nov 10 Public Release

23
Changes to the Report Card

• Only three overall rating categories
• Outstanding
• Satisfactory
• In Need of Improvement
• Graduation Rates used instead of dropout rates.
• The growth model will be incorporated into the
  rating formula.

24
New Report Card Values

• We reward schools for
• students meeting standard
• students exceeding standard
• students showing growth
• closing the achievement gap
• We have been developing this model since Summer
  2008 with input from key stakeholder advisory
  groups

25
The New Achievement Index

• Points awarded according to
• 133 points for Exceeds
• 100 points for Meets
• 100 points for Did Not Meet, but Met Growth
• Use the same students as are used for AYP
• includes extended assessments
• includes the 1 rule for extended assessments
• At high school we will use growth in school
  performance as the growth measure, and will give
  partial credit for nearly meets and low.

26
Overall Rating

• Subgroups are not rated, only a holistic school
  rating is given.
• To address the Achievement Gap, a schools
  Achievement Index is a weighted average the
  achievement of subgroups.
• Subgroups in the weighting include All Students
  plus subgroups with an historic statewide
  achievement gap.
• Low Attendance/Graduation and Participation can
  lower the school rating.

27
Detail Sheet Preview
28
Likely Impact on School Ratings

• About half of the formerly strong schools will
  be rated as Outstanding
• Low and Unsatisfactory schools generally move
  to Needs Improvement
• The following table shows how schools would have
  been rated in 07-08 if the new formula had been
  applied.

29
Potential Changes Impact on Ratings
30
Cohort Graduation Rate
31
Timeline

• June 2009 -- The 0708 Informational Cohort Rate
  provided to districts.
• Summer 2009 -- ODE will submit for approval from
  the US Department of Education
• Summer 2009 -- begin data cleanup for the 0809
  cohort
• Spring 2010 -- 2008-09 Cohort rate will be
  reported
• Fall 2010 -- 2008-09 Cohort rate will be used in
  AYP determinations

32
What is the Cohort Rate?

• The Cohort Rate is the percent of students who
  receive a regular diploma within four years of
  entering high school.
• The Cohort Rate tracks students over the course
  of 4 years starting with their first year in high
  school.
• Each member of a cohort will be assigned a final
  outcome at the end of the four years.

33
Data Sources and Process

• The Cohort rate uses Resident Institution only.
• ODE uses records from ADM, Membership, Early
  Leavers, and High School Completers to assign
  students to a probable cohort.
• Districts confirm the cohort of the student.
• Final outcomes are determined from the most
  recent events in our databases.
• Students are assigned to their most recent
  diploma granting institution.

34
Who is included?

• The unadjusted cohort is comprised of any
  student, enrolled in an Oregon public school by
  the end of the 2007-08 school year, who first
  entered high school in 2004-05 in any school
  inside or outside Oregon.
• The adjusted cohort is comprised of the
  unadjusted cohort minus students who died,
  entered the home schooling system, or transferred
  out to a diploma granting institution outside the
  Oregon Public Education system as their final
  high school educational setting.

35
Sample Statewide Data
36
Federal Requirements

• State must set a graduation Goal
• This need not be 100
• State must set yearly targets that show
  significant progress toward the goal
• Minimum group size should not exceed the minimum
  group size for participation (40).
• Margin of Error is unlikely to be approved.

37
Federal Flexibility

• AYP reports may average data over two years, or
  use the most recent year, whichever is higher.
• States are responsible for developing subgroup
  membership rules.
• Four-year rate can include summer graduates.

38
Available Options

• State may develop an extended-year rate.
• It may not supplant the four-year rate.
• Targets must be more aggressive (goal is the
  same).
• Targets for individual schools may vary
• States can implement a Safe Harbor provision

39
The following slides represent ODE leanings on
the required policy decisions. These are
subject to revision.
40
How to Meet on Graduation

• Schools/subgroups can meet on AYP
• Graduation through
• Meeting the target for the four-year rate based
  on a two-year average or the most recent year or
• Decreasing the of students not graduating by
  10 from one year to the next or
• Meeting the target for the Extended rate based
  on a two-year average or the most recent year

41
Graduation Goal and Targets

• Graduation Goal of 90
• Targets with a 2 increase from current targets
  to the goal in 2020-21.

42
Minimum Cell Size

• ODE recommends the minimum be a total cohort size
  of 20 (an average of 10 for each year).
• Distribution of Single Year Cohort Size
• Less than 10 29 schools
• 10 to 19 30 schools
• 20 to 39 48 schools
• 40 or more 208
• -- These schools are typically rated on
  attendance.

43
Subgroup Membership

• Race/Ethnicity
• Based on the last student record.
• Special Education, Economically Disadvantaged and
  Limited English Proficient
• Based on whether the student was classified in
  these subgroups at any time during high school

44
Contact Information

• For more information on the Growth Model or the
  Report Card, contact
• Jon Wiens at
• jon.wiens_at_state.or.us
• Phone [phone]
• or visit the Growth Model Web Page at
• http//www.ode.state.or.us/search/page/?id2495