Development of a Math Screening Assessment on a Districtwide Basis

1
Development of a Math Screening Assessment on a
Districtwide Basis

• Washington Educational Research Association
• Annual Conference
• December 5-7 2007

Mike Jacobsen-Assessment and Curriculum
Director Andy McGrath-Principal Glacier Middle
School White River School District 360-829-3820 mj
acobse_at_whiteriver.wednet.edu
2
By The End of This Presentation You Will

• Understand how the district implements a K-10 CBM
  reading assessment system
• Understand how the WRSD developed a math screener
• District-wide focus
• Establish a committee
• Pilot process
• Full implementation
• Fall, winter spring data 06-07
• Next steps

3
Basic Definitions

• CBMCurriculum Based Measurement
• Developed Initially at University of Minnesota
  Institute for Research on Learning Disabilities
• Measures students progress in basic skills using
  existing curriculum
• Psychometrically sound
• ORFOral Reading Fluency
• What is measured is students ability to read out
  loud, accurately and fluidly
• DIBELSDynamic Indicators of Basic Early Literacy
  Skills
• Researchers from the University of Oregon coined
  the phrase

4
What is CBM?

• Standard, simple, short duration fluency measures
  of reading, spelling, written expression and
  mathematics computation
• WRSD Reading CBM is very similar to DIBELS with
  one exception
• WRSD Math Screener is different than DIBELS in
  math
• In reading CBM is oral reading fluency
• Measures vital signs of student achievement
• Academic thermometer

5
Big Ideas About CBM

• Extensive data supporting validity of use as a
  measure of basic skills
• Principle use is in formative evaluation
• Sensitive to changes in performance due to
  instruction
• Easy to use within classrooms
• Brief
• Repeatable

6
(No Transcript)
7
ORF and Other Reading Tests

• 1999- 3rd Grade Qualitative Reading Inventory to
  3rd Grade ORF.89
• 1999-3r Grade ITBS to 3rd Grade ORF.64
• 1999-2th Grade Gates-MacGinitie to 2nd Grade
  ORF.84
• 1999-3rd Grade Gates-MacGinitie to 3r Grade
  ORF.77
• 1999-4th Grade Gates-MacGinitie to 4th Grade
  ORF.64
• 1999-5th Grade Gates-MacGinitie to 5th Grade
  ORF.86

8
ORF and WASL Relationships

• 1998- 4th Grade WASL to 5th Grade ORF.70
• 1999-4th Grade WASL to 4th Grade ORF.51
• 2003-6th Grade ORF to 7th Grade WASL.68
• 2000-4th Grade ORF to 4th Grade WASL.66
• 2002-4th Grade ORF to 4th Grade WASL.65

9
ORF and WASL Relationships
10
CBM ORF/WASL
11
(No Transcript)
12
Why Assess Computational Fluency?

• Many of the difficulties children have in
  arithmetic result from not understanding number
  ideas supposedly learning at an earlier time
• Engelhart, Ashlock Wiebe, 1984
• In most cases the precision and fluency in the
  execution of the skills are the requisite
  vehicles to convey the conceptual understanding.
• H. Wu, 1999

13
White River School District Assessment Process

• Implemented during the 98-99 school year for K-6
  Reading
• 6th-8th grade added 2002
• 9th/10th grade added 2005
• Implemented during the 2006-2007 school year for
  1-10 Math screener
• Kindergarten students, initial sound fluency,
  letter names and segmenting phonemes
• Grades 1-10 orally read passages from appropriate
  grade level material
• Conducted three times per year during September,
  January and May

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
Background of Development of the Math Screener
District Learning Improvement Planning

• Established Fall of 2005
• Approximately 30 members, teachers, building
  administrators, central office administrators,
  parents and school board members
• Each building had a stipend position for a
  teacher who served as DLIP coordinator
• Met monthly during the 05/06 school year
• The first meeting was on structure and goals,
  research on effective schools and role of the
  district

21
Background District Learning Improvement Planning

• Established Fall of 2005
• Approximately 30 members, teachers, building
  administrators, central office administrators,
  parents and school board members
• Each building had a stipend position for a
  teacher who served as DLIP coordinator
• Met monthly during the 05/06 school year
• The first meeting was on structure and goals,
  research on effective schools and role of the
  district

22
Background District Learning Improvement Planning

• The second meeting focused on district-wide
  information using the data carousel format
• WASL trend data-desegregated
• ITBS
• CBM
• Demographics
• Safe and Civil Surveys
• Nine Characteristics
• Healthy Youth Survey
• Sports and Arts program participation
• Curriculum alignment
• Professional development

23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Background District Learning Improvement Planning

• Used data from the carousel process to identify
  three major focus areas
• Professional development
• Curriculum alignment-math
• Math
• Each focus area had co-chairs
• Every member of the district learning improvement
  team was on one of the focus area committees
• Outcome oriented

27
Math Committee

• District Math TOSA Kathie Ross and Andy McGrath
  Co-chaired the Math Committee
• Goal To produce a math assessment that will
  reliably predict a students success on the WASL
  (not diagnostic)
• To produce an assessment that can be given in
  20-30 minutes and can be graded in a timely
  manner without added cost

28
Math Committee

• To Achieve This Goal
• We added teachers to the committee from each
  level primary, intermediate, middle and high
  school
• Committee Makeup
• 3 Administrators
• 7 Teachers
• 1 Central Office
• 2 Parents

29
Math Committee

• ASSESSMENT DEVELOPMENT
• Committee met for about 2 months discussing the
  makeup of the assessment
• Assessment Structure
• 20 Total Questions
• 12 Computation
• 8 Applied Problems
• Single Number Answer
• Reading fluency assessment already established in
  district
• Reviewed Fuchs and Fuchs-Monitoring Basic Skills
  Progress-2nd Ed.
• Reviewed Ken Howells et all- Multilevel Academic
  Skills Inventory-Revised
• Next step split subcommittee into three groups
• Elementary
• Middle
• High School

30
Math Committee-Assessment Cont.

• The Groups using the GLEs as a guide developed a
  draft assessment for each grade level
• Assessments were brought back to full committee
  to be discussed and edited
• Developed assessments for grades 2 10
• Assessment give three times a year in conjunction
  with reading assessment

31
Sample Page 4th Grade Computation
32
Sample Page 4th Grade Applied Problems
33
Pilot Process

• IMPLEMENTATION
• An assessment for each grade level completed by
  April 2005
• Piloted last May with volunteer classrooms at
  least two per grade level
• Pilot results to Assessment Office analyze math
  assessment and reading fluency to see if this
  would be a good predictor of WASL success
• If the assessment proved to be an accurate
  predictor of WASL success then implement
  district wide Fall 06

34
Pilot Process

• Manila envelope provided to each pilot teacher
• Directions for Administration
• Instructions for Scoring
• Student Response Sheets
• Test Key
• Copies of student response forms provided to each
  teacher
• 624 students grades 1-8 participated
• Statistically strong relationships with WASL math
  and spring oral reading fluency demonstrated

35
Pilot Results