Raising the Bars: 68 Week Reading Skills Assessments and the CST ELA Paper presented at Reading Firs

1
Raising the Bars6-8 Week Reading Skills
Assessments and the CST ELA Paper presented at
Reading First Local Education Agency
MeetingsApril 26, SacramentoMay 4, Los Angeles
Dr. Jim Parker Research and Evaluation Paramount
Unified School District
2
Outline of Presentation

• Describe our demographic population and our
  academic achievements
• Who we are
• What weve done
• Overview of our interim assessments and their
  relationship to the CST ELA
• Raising the bar with our interim
  assessments---what weve done

3
District Profile Paramount Unified

• Where Paramount is
• South Central Los Angeles County, surrounded by
  Lynwood, Compton, Long Beach, Bellflower and
  Downey
• District demographics
• 16,500 students
• 86 receiving Free/Reduced Lunches (NSLP)
• 48 English Learners
• Annual school mobility rates in the 10-25 range
• 82 Hispanic 12 African-American
• About half the students have parents without a
  high school degree between them three quarters
  have parents with no more than a h.s. degree (no
  college) and about 10 have a parent with a
  college degree.
• Schools
• 13 K-8 schools, one K-3, one Gr. 4-8
• One comprehensive high school

4
(No Transcript)
5
(No Transcript)
6
Times New Roman
l
7
(No Transcript)
8
(No Transcript)
9
What did we do to make our gains?
10
Determined Implementation Reading First Year 1
High RFAI, High Implementation
Index
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxxx
11
(No Transcript)
12
Assessment and Data Analysis

• Reading Skills Assessments (RSAs) every 6-8
  weeks
• An On-line Assessment Reporting System (OARS) to
  track our data and produce reports
• A school site reading coach at each school and
  set aside planning time to discuss results
• Districtwide goals and objectives driven by, and
  attainment validated by, data

13
Focus on one aspect of data using interim
assessments to project success on high stakes
year end state assessments.How do your interim
assessments relate to those high stakes tests?
14
Taking the RSA slices and piecing together a
better predictor of the CST ELA The Composite
Index or Whole Test Index
For example, TEST 2
The Composite Index combines only subtest scores
on a single RSA assessment, e.g., Test 2.
15
Advantages to Whole Test Index (WTI)

• Whole test index looks more like the CST ELA
  (with vocabulary, comprehension, spelling, etc.,
  in one index)
• Each subskill could have been, but was not,
  weighted so that the weight of each in the WTI
  was
• equal to the percentage of items it contributed
  to the WTI, or
• equal to the percentage of the CST ELA test made
  up from the similar cluster (e.g., if 20 of CST
  is vocabulary, vocabulary subskill could be
  weighted 20 in WTI)
• More items brings greater discriminatory power
• one scale, such as fluency, or one subskill test
  with 10 items does not discriminate well
  differences in a 70-80 item CST test
• another 50-60 item WTI test does discriminate
  better.

16
Accumulating the RSA results on all tests to
date to get a better predictor of CST ELA A
Cumulative Index (CI) Measure
TEST 2
TEST 1



TEST 3
and so on.
17
Correlations of RSA Subtests/ Composite Tests in
Predicting CST ELA, 2003
18
Correlations of RSA Subtests/ Composite Tests in
Predicting CST ELA, 2003
19
Average Fluency vs CST ELA, 2002-03
nt
National Benchmarks for Reading Fluency are below
average wcpm of students Proficient on CST ELA.
20
(No Transcript)
21
2004 and 2005 Updates
22
2003
2004
23
2005
24
2003
2004
25
Using a Schools Cumulative Index Gains/Losses
to Predict CST ELA Gains/Losses
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
Raising the Bars

• Setting Higher Benchmarks for RSA Subskills and
  for the Cumulative Index

33
Type I vs Type II Errors

• Type I Error Thinking a student is not
  Proficient, when in fact he is.
• Type II Error Thinking a student is Proficient,
  when in fact he is NOT.
• Typical, commonsense assumptions
• Grade Level Benchmark Proficient
• No !!!

One Tests Benchmark is NOT another tests
Proficient.
34
Setting the benchmark bar too low causes more
Type 2 errors setting the benchmark bar too
high causes more Type 1 errors.
35
2002-03 Data for ParamountThe 2003 CI and
Percentage Proficient on the 2003 CST ELA

• The CI is a continuous variable scale, 0-10, so
  that it averages all RSA tests across each grade
  (including Average Fluency)

36
Likelihood of Scoring at Each Level of the
English/Language Arts California Standards Test
by Unit 4 RSA Cumulative Index Level, 2004
37
Likelihood of Scoring at Each Level of the
English/Language Arts California Standards Test
by Unit 4 RSA Cumulative Index Level, 2003 to
2005
38
English Language Arts CST Results, Paramount
Unified, 2001 to 2005 Percent Proficient or
Higher
39
of Students Scoring at Least 7.0 on the
Cumulative Index after Unit 4, 2003 thru 2005
40
New Benchmark Levels for Grades 1-5

• Kindergarten Varies by subskill
• Grades One to Three 8 for each subskill
• (Writing still at 3 out of 4)
• Cumulative Index
• (Gr 2-3) 7.5
• Grades Four and Five 7 for each subskill
• (Writing still at 3 out of 4)
• Cumulative Index 7.0 

41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
What are the outcomes from raising the benchmark
bars?

• Closer match between the students who are at
  benchmark and who will be Proficient on the CST
  ELA.
• Identifying students just short of hitting
  benchmark, means we can better identify them as
  most likely to fall just short of Proficient, and
  do interventions.
• Less confusion between notions of benchmark,
  grade level and proficient, as they are now more
  alike.

46
What are the outcomes from raising the benchmark
bars?

• Theory Higher, but more realistically set,
  benchmarks mean we can aim towards goals in
  lockstep with the CST ELA and so bring more
  students to the higher levels we know we need to
  attain for Proficiency.
• Still too early to tell if raising benchmarks has
  accelerated performance
• Benchmarks raised only a few months ago analysis
  just begun
• Accelerating gains in at least one grade
• Apparently gains in all grades over the year
  before

47
Similar Students, Different ResultsWhy Do Some
Schools Do Better?(California elementary schools
serving largely low income students)

• Schools tend to score higher on the API that work
  on
• Prioritizing student achievement.
• Implementing a coherent, standards-based
  curriculum and instructional program.
• Using assessment data to improve student
  achievement and instruction.

48
Similar Students, Different ResultsWhy Do Some
Schools Do Better?

• Schools tend to score higher on the API that work
  on
• Ensuring availability of instructional resources.
• Principal leadership in the context of
  accountability-driven reform is being redefined
  to focus on effective management of the school
  improvement process.
• District leadership, accountability, and support
  to influence student achievement.