CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model

1
CBM, Charting, Progress Monitoring, and Data
Based Instructional Decision Making in a RTI
Model

• Tom Jenkins, Ed.D.
• Educational Consultation Services, LLC
• Wilmington, NC

2
What you can expect today

• CBM, CBA, CBE
• Instruction on progress monitoring
• Examples and hands-on activity
• Final thoughts and conclusions

3
What is CBM?

• Curriculum-based measurement
• Data collection tools derived directly from the
  curriculum that student is expected to learn

4
CBM

• CBM is believed to reduce the gap between
  assessment and instruction
• Aides teachers in generating superior student
  achievement
• Improved communication
• Higher level of sensitivity
• Enhancement of the database
• Administration time is shorter
• More cost effective

5
CBM

• The simplicity of CBM allows for quick and easy
  peer referencing
• Normative data can be collected
• This allows for comparison of a students
  performance to his/her actual peer group
• More representative geographically, culturally,
  ethnically, and has been exposed to similar
  instructional environment

6
CBM

• CBM has been shown to posses high levels of
  reliability
• 42 one-minute CBM type assessments in reading,
  math, and written expression for grade K-5 were
  found to have reliability coefficients between
  .90-.99 with just three one-minute
  administrations (Jenkins, 2002)

7
CBM

• Discriminant Validity
• Several studies have demonstrated the ability of
  CBM to differentiate between students receiving
  special education services, students receiving
  Chapter 1 services, and students not receiving
  any of those services (Deno, Marston, Shinn, and
  Tindal, 1983 Marston and Deno, 1982 Shinn and
  Marston, 1985 and Shinn, Tindal, Spira, and
  Marston, 1987).

8
CBM Procedures

• Remember different modalities
• Given the nature of reading probes they must be
  administered individually
• Math and written expression may be administered
  in a group setting

9
CBM Procedures

• Scoring
• One minute administration time except for written
  expression
• Number ID is scored corrects for minute however,
  math computations are scored digits correct per
  minute
• Written expression is scored according to correct
  sequences

10
Example

• Administration example
• Work in pairs
• Each person gets a chance to adminster

11
CBM Procedures

• Written expression examples

12
Progress Monitoring

• Basic Principle 8 of PSM
• Progress monitoring an essential aspect of the
  intervention phase
• Basic Principle 9 of PSM
• Decision making in regards to the effectiveness
  of an intervention is based on analysis of
  progress monitoring data in relation of goal

13
Progress Monitoring

• Essential for four reasons
• There is no guarantee that interventions will be
  successful, thus the intervention must be
  tested to evaluate effectiveness
• Increased emphasis of specific outcomes for
  students, data base must be generated to guide
  intervention decision making
• Pre/post testing has be shown to be unreliable
  (small amount of data) and provides too little
  data to allow for instructional decision making
  progress monitoring allows for evaluation of
  level of performance and rate of learning
• Research has shown that progress monitoring is
  associated with improved educational outcomes

14
Progress Monitoring

• Definition
• Frequent and repeated data collection (no less
  than 2-3 times a week) and analysis of student
  performance
• Data is collected during intervention and
  provides basis for intervention effectiveness

15
Progress Monitoring

• Essential components that must be in place for
  successful progress monitoring
• A well-defined behavior
• A measurement strategy
• Identification of students current level of
  performance (baseline)
• Intervention
• Goal
• Graph
• Decision-making plan

16
Well Defined Behavior

• Target behavior, observable, measurable, and
  specific
• Focus on enabling skills
• Skills that are prerequisite skills for more
  complex skills
• Deficiencies in enabling skills often adversely
  affects performance on global assessments

17
Well Defined Behavior

• Enabling skills for reading
• Phonemic awareness
• Alphabetic understanding
• Fluency
• Sight words
• Comprehension

18
Well Defined Behavior

• Enabling skills for math
• Number sense
• Facts
• Computation
• Applications
• Problem solving
• Enabling skills for written expression
• Mechanics
• Expression

19
Well Defined Behavior

• Enabling skills for behavior
• Social skills
• Work completion
• Compliance
• Problem solving skills

20
Measurement Strategy

• Frequent and repeated collection of data
• Time and cost efficient
• Sensitive to changes over short periods of time

21
Current Level of Performance (Baseline)

• Gathered prior to intervention
• Repeatable
• Provides comparison for progress data
• Helps set goal
• Median score why?

22
Intervention

• Match intervention to problem
• Humans tend to employ interventions with which
  they are comfortable instead of intervention that
  the student needs
• Intervention should be developed with the
  expectation that it will be altered in some way
  as a result of the progress monitoring data
• No intervention works all of the time for every
  student

23
Goal

• Standard against which progress can be compared
• Allows for aimline to be established
• Possible goals
• Norms
• Percentile cutoff
• Realistic growth rates
• Ambitious growth rates
• Minimum celeration
• Local growth rates

24
Computing Growth Rates

• Winter Norm minus Fall Norm, divided by number of
  weeks between norming projects
• Gives you a growth expectancy for each week of
  school year
• Allows for obtaining students baseline then
  monitoring progress while comparing to growth
  expectancy

25
Computing Growth Rates

• Example first grade, ID words in sentences
• Winter Norm minus Fall Norm, divided by number of
  weeks between norming projects
• 67.9119-35.2694 / 10 3.2643 words per week

26
Growth Rates based on research by Doug and Lynn
Fuchs

• Realistic Growth Rates
• Gr 1 2 words/week
• Gr 2 1.5 wrds/week
• Gr 3 1 words/week
• Gr 4 .9 words/week
• Gr 5 .5 words/week

• Ambitious Growth Rates
• Gr 1 3 words/week
• Gr 2 2 words/week
• Gr 3 1.5 wrds/week
• Gr 4 1.1 wrds/week
• Gr 5 .8 words/week

27
Goal

• 1.25 minimum celeration, Precision Teaching

28
Graph

• Provides a visual representation of a large
  amount of data
• A visual representation of students acquisition
  of skills and allows for easier analysis of
  progress
• Semi-log chart
• Equal interval charts can misrepresent data,
  depending on how axis is quantified
• Equal interval charts assumes equal amounts of
  progress between all data points
• Precision Teaching

29
Decision Making Plan

• Facilitates interpretation of data
• Should include
• Rule for raising performance goal 6/4
  consecutive data points above the aimline
• Rule for altering the intervention due to lack of
  progress 3/4 consecutive data points below the
  aimline
• Deno/Allison

30
Decision Making Plan

• If the decision is to adjust an intervention,
  small changes or refinements are recommended
  before major changes
• However, changes should be substantial enough
  that it has a possibility to result in improved
  student performance
• If making an adjustment, do not make two at the
  same time. It may result in the team being unable
  to determine what caused increased student
  performance

31
Decision Making Plan

• As student performance approaches goal, team must
  decide
• Raise the goal
• Begin work on another target behavior
• Discontinue PSM process
• If in level four consider
• Discontinuation of EC services

32
Charting Activity

• A means of interpreting large amounts of data
• Allows for relating performance trends to desired
  performance
• Research has shown the charting also facilitates
  positive student outcomes (Fuchs, 1989)

33
Charting Activity

• Definition
• Visual depiction of the students performance
  data, relative to the goal and aimline
• Includes baseline data, goal, aimline, and
  progress monitoring data

34
Charting Activity

• Procedures
• Write the prediction/goal statement
• Depict baseline data collection phase on the
  chart, indicate the median score, identify with a
  heavy dark line
• Depict the goal at the end of the anticipated
  intervention phase
• Depict the aimline by connecting the baseline
  median with the goal, this gives you expected
  rate of progress

35
Charting Activity

• Activity
• Plot the baseline data
• Monday 14
• Tuesday 10
• Wednesday 6
• Thursday 5
• Friday 10
• Plot the median and signify with a heavy dark
  line

36
Charting Activity

• Plot the goal at the end of eight weeks
• 40
• Indicate the aimline

37
Charting Activity

• Performance trends should be analyzed
  periodically
• Trend above aimline raise the goal
• Trend below aimline adjust intervention
• If changes are made to the intervention, indicate
  change on the graph with a squiggle line
• Describe the changes on the back of the chart
• This allows for understanding of specific
  instructional adjustments that were
  successful/unsuccessful

38
Charting Activity

• Plot the first two weeks progress monitoring data
• Week One
• Tuesday 16
• Thursday 14
• Week Two
• Monday 10
• Wednesday 18
• Friday 16
• Make an informed decision regarding the
  effectiveness of the intervention

39
Charting Activity

• Remember
• Rule for raising performance goal 6/4
  consecutive data points above the aimline
• Rule for altering the intervention due to lack of
  progress 3/4 consecutive data points below the
  aimline
• Deno/Allison

40
Charting Activity

• Plot progress monitoring data for next two weeks
• Week Three
• Tuesday 14
• Thursday - 10
• Week Four
• Monday 15
• Wednesday 16
• Friday 16
• Make an informed decision regarding the
  effectiveness of the intervention

41
Charting Activity

• Phase two or three of the intervention?
• Plot the progress monitoring of the next two
  weeks
• Week Five
• Tuesday 24
• Thursday - 26
• Week Six
• Monday 26
• Wednesday 24
• Friday 26
• Make an informed decision regarding the
  effectiveness of the intervention

42
Charting Activity

• Phase two or three of the intervention?
• Plot the progress monitoring of the next two
  weeks
• Week Seven
• Tuesday 28
• Thursday - 30
• Week Eight
• Monday 32
• Wednesday 36
• Friday 38
• Make an informed decision regarding the
  effectiveness of the intervention
• Final decision?

43
Charting Activity

• More information than you wanted to know
• Trend lines
• Line that you draw through a series of data
  points that represents the students actual rate
  of progress
• If trend line slope is flatter than aimline
  slope, then adjust intervention
• If trend line slope is steeper than aimline
  slope, then adjust goal
• If slopes are the same, make no change

44
Charting Activity

• Trend line procedures
• Draw vertical line in middle of graph, half data
  points on one side, half on the other if odd
  number of data points, put line through middle
  point
• Draw a vertical line in first half of data to
  separate data half and half
• Draw a vertical line in second half of data to
  separate data half and half

45
Charting Activity

• Trend line procedures
• Draw a horizontal line through median of first
  half of data form an intersection with vertical
  line
• Draw a horizontal line through median of second
  half of data form an intersection with vertical
  line
• Connect the two intersections
• Make your decision, is it the same decision?

46
Thoughts and Conclusions

• What if we took it a step further?
• What if we used the intervention/progress
  monitoring/charting process to move students
  towards a CBM reading fluency score that we knew
  predicted success on EOGs and EOCs?

47
Thoughts and Conclusions

• Espin et al, 2002 found correlation between CBM
  reading fluency scores and Minnesota grade 8
  state standards test scores to be .78
• Grade 5 correlation between CBM reading fluency
  scores and Minnesota Comprehensive Assessments -
  .77

48
(No Transcript)
49
Thoughts and Conclusions

• Similar results were found in Minnesota for
  grades 3 and 5
• Similar results have been found in different
  states

50
(No Transcript)
51
Thoughts and Conclusions

• Similar results have been found in Oregon and
  Washington

52
Thoughts and Conclusions

• What does all this mean?
• Implement a school-wide model of progress
  monitoring
• Screen all students in the fall
• Identify students with CBM fluency scores that
  make them at risk on the standards based
  testing
• Intervene and monitor progress of the students
  at risk move them towards a CBM reading
  fluency score that predicts a higher likelihood
  of success on the standards based testing

53
Thoughts and Conclusions

• Goals can be set based on local norming data
  find CBM reading fluency for those that did meet
  standard
• Growth rates realistic or ambitious
• Fluency benchmarks set by research (Deno)
• 1st grade 60 wpm
• 2nd grade 90 wpm
• 3rd grade 120 wpm
• 4th grade 130 wpm
• 5th grade 140 wpm
• 6th grade 150 wpm

54
Thoughts and Conclusions

• How can I progress monitor with all the other
  things that I have to do?
• School improvement occurs when focus is placed on
  student improvement outcomes, priorities are
  going to have to be changed so that progress
  monitoring is seen as just as important as
  instruction

55
Thoughts and Conclusions

• We have standards based tests to measure student
  progress, why do more?
• These tests are usually administered once a year
  and show student success in meeting a criterion.
  Progress monitoring enables learning, shows if a
  student is making progress towards success on
  standards based tests, and allows for informed
  instructional decision making that helps the
  student reach the goal of success on standards
  based tests.

56
Questions?

• Dr. Tom Jenkins, Director
• Educational Consultation Services, LLC
• Wilmington, NC
• [phone]
• Fanofstel_at_aol.com
• www.educationalconsultationservices.com