Curriculum Based Measurement and Language Proficiency in English Language Learners

1
Curriculum Based Measurement and Language
Proficiency in English Language Learners

• Ella Farmer, M.Ed. Laura Swanlund, M.Ed, NCSP
  Kathy Pluymert, Ph.D, NCSP
• Community Consolidated School District 15,
  Palatine IL
• NASP 2010

2
The need for this study

• Large, elementary district with a diverse student
  body
• CBMs introduced at the district level in the
  2008-2009 school year as part of the transition
  to RTI
• We wanted to see if CBMs were a reliable
  indicator of growth rates for ELLs
• We wanted to understand the rate of growth for
  ELLs in relation to instruction

3
What led up to this study

• Data showed that some ELL students were not
  transferring literacy skills into English
• By 4,5,6th grade the students were not making
  progress on standardized achievement tests
  (Measures of Academic Progress)
• Performing poorly in the classroom
• We need a process that looks at dual discrepancy
  compared to the correct peer group.
• Teachers and administrators questioned whether
  CBMs were a reliable measure for ELLs

4
Research on this topic

• Students who enter kindergarten with limited
  English proficiency lag behind their peers in
  reading achievement through 5th grade.
  Controlling for risk factors such as SES reduces
  the effect of LEP status on achievement (Kieffer,
  2008)
• Reading CBM assessments have been found to be
  reliable, valid, and sensitive measures to use
  with bilingual Hispanic students (Baker Good,
  1995)
• Moderate correlations have also been found
  between ELL students performance on reading CBM
  measures and a state standards test (Wiley
  Deno, 2005)
• When using both Spanish and English CBMs with
  bilingual students, rates of improvement in both
  languages were significantly lower for ELLs than
  that which is seen in the overall population
  (Dominguez de Ramirez and Shapiro, 2006)

5
Our research questions

• Is there a significant difference in initial
  R-CBM scores or rate of growth when comparing
  ELLs and English-speaking peers?
• Is there a correlation between English language
  proficiency and R-CBM or rate of improvement?
• Is there a difference in initial R-CBM score or
  rate of growth depending on SES, reading
  intervention, or type of ESL instruction?

6
The data

• Large, suburban school district with 15
  elementary schools
• Four schools selected based on concentration of
  ELLs and available CBM data
• N 1,197 students, Grades 1-3
• Measures included
• R-CBM scores for fall, winter, and spring
• ROI from fall to spring
• English Language Proficiency (ACCESS for ELLs
  composite score)
• SES (free or reduced lunch status)
• ELL status
• Reading intervention
• Type of ESL instruction (Bilingual or Resource)

7
Descriptive data
8
Average R-CBM Scores and ROI
9
Correlations between R-CBM and ACCESS
10
HLM

• What we did
• For each grade looked at R-CBM and predictor
  (i.e. SES and ELL status)
• Looked at initial CBM status and rate of growth
• Intercept for each grade Average fall R-CBM
  score
• Slope for each grade Rate of growth
• Looked at the above based on SES, ELL status, and
  Reading Intervention
• Separate analysis for ELL students only, looking
  at ESL instruction, Reading Intervention, and SES
  as predictors
• Because there were moderate to strong
  correlations between ACCESS scores and R-CBM,
  this variable was not added as a predictor

11
Level 1 Analysis
At all levels there was significant variation
around the intercept and slope, which means that
since there is variance we can look at factors
that may contribute to the variance, such as SES
status, ELL status, and intervention status.
12
Fall Initial Score difference from students with
higher SES and non-ESL by grade
13
Fall Growth Rate Score difference from students
with no intervention, higher SES, and non-ELL by
grade
14
Fall Initial Score difference from ELL students
in Resource and higher SES
15
Fall Growth Rate Score difference from ELL
students in Resource and higher SES
16
Summary of the findings

• HLM- All students
• Students in grades 1-2 show significant growth on
  CBM measures. This is true of ELL students as
  well as English speaking students. While most
  students in grade 3 demonstrate significant
  growth over the school year, some do not.
• ELL students and lower SES students have lower
  initial (Fall) CBM scores at all grade levels,
  when compared to peers with no risk factors.
• ELL students improve at a lower rate than their
  peers in first grade. However, this reverses and
  their rate of improvement is significantly higher
  than their peers in second grade. In third
  grade, there is no significant difference in
  rates of improvement when comparing ELL students
  and English-speaking students.
• In third grade, students who are lower SES
  improve at a higher rate than their peers who are
  not at risk.
• Students who are receiving reading intervention
  have a significantly lower rate of improvement
  than do their peers who are not receiving
  intervention.
• For all students in all grades, correlations
  between slope and intercept were weak to
  moderate.

17
Summary

• HLM- ELL Students
• ELL students from lower SES backgrounds in grade
  1 have lower initial R-CBM scores.
• ELL students who are lower SES in grade 2 have
  significantly lower rates of growth than their
  ELL peers of higher SES.
• ELL students in grade 1 who receive reading
  intervention have significantly lower rates of
  growth than their ELL peers who are not receiving
  intervention.
• In grades 2 and 3, ELL students who are in a
  bilingual program have lower initial R-CBM scores
  than do their peers who receive ESL resource
  support. This is consistent with the finding
  that R-CBM scores are highly correlated with
  ACCESS scores.
• There are no significant differences in ROI
  depending on the type of ESL program (bilingual
  or resource) that students receive.

18
Implications for ELLs

• When looking for a dual discrepancy under RTI,
  ELL students initial R-CBM scores should be
  compared to ELL peers.
• A lower initial Fall R-CBM score is expected for
  this group when compared to English Speaking
  peers, therefore, it is best to compare initial
  scores to ELL peers
• Compare rate of improvement to ELL peers
• The results suggest that in all grade levels ELL
  students should be making consistent progress on
  the R-CBM during the year
• ELL progress should be considered in terms of
  ACCESS, SES status, ESL status, and academic year

19
Limitations

• Did not examine early literacy measures for Grade
  1. Research suggests that NWF may be a more
  reliable and valid measure than RCBM (Vanderwood,
  Linklater, Healy, 2008)
• ELL students in this study were primarily
  Spanish-speaking
• Did not examine predictive validity
• Did not examine Spanish RCBM scores

20
Use of this data for our district

• Triangulation of data
• Administration, teachers, and specialists
  received training on how to integrate multiple
  data sources for ELL students
• Used of the results in order to determine the ELL
  peer comparison expected scores and growth rates
• Examined the curriculum and appropriate tier 2
  interventions
• Data analysis occurred at multiple levels
• Program evaluation
• School-wide data digs
• Quarterly grade level data meetings
• Individual problem solving

21
Future directions

• Need to examine predictive validity of R-CBM and
  correlation with other measures, such as MAP and
  ISAT
• Need to examine data for grades 4-8
• Need to examine early literacy data for grades
  K-1

22
References

• Baker, S. K., Good, R. (1995). Curriculum-based
  measurement of English reading with bilingual
  Hispanic students A validation. School
  Psychology Review, 24(4), 561.
• De Ramírez, R. D., Shapiro, E. S. (2006).
  Curriculum-Based Measurement and the evaluation
  of reading skills of Spanish-speaking English
  language learners in bilingual education
  classrooms. School Psychology Review, 35(3), 356.
  
• Kieffer, M. J. (2008). Catching up or falling
  behind? Initial English proficiency, concentrated
  poverty, and the reading growth of language
  minority learners in the united states. Journal
  of Educational Psychology, 100(4), 851-868.
• Vanderwood, M. L., Linklater, D., Healy, K.
  (2008). Predictive accuracy of Nonsense Word
  Fluency for English language learners. School
  Psychology Review, 37(1), 5.
• Wiley, H. I., Deno, S. L. (2005). Oral reading
  and maze measures as predictors of success for
  English learners on a state standards assessment.
  Remedial and Special Education, 26(4), 207.