Graphing, Calculating, and Interpreting Rate of Improvement

1
Graphing, Calculating, and Interpreting Rate of
Improvement

• Caitlin S. Flinn, Ed.S., N.C.S.P.
• Andrew E. McCrea, M.S., N.C.S.P.
• PaTTAN RtII Institute
• June 14, 2010

2
Objectives

• There needs to be a standardized procedure for
  calculating RoI
• Were proposing a method using Simple Linear
  Regression

3
Overview

• Importance of RoI
• RoI Foundations
• A Need for Consistency
• Calculating RoI
• Individual Student Graphs
• Programming Excel

• Decision Making
• Grounding the Data
• Interpreting Growth
• Individual Student
• Student Groups
• Considerations
• Resources

4
Importance of Graphs

• Vogel, Dickson, Lehman, 1990
• Speeches that included visuals, especially in
  color, improved
• Immediate recall by 8.5
• Delayed recall (3 days) by 10.1

5
Importance of Graphs

• Seeing is believing.
• Useful for communicating large amounts of
  information quickly
• A picture is worth a thousand words.
• Transcends language barriers (Karwowski, 2006)
• Responsibility for accurate graphical
  representations of data

6
Skills Typically Graphed

• Reading
• Oral Reading Fluency (ORF)
• Word Use Fluency (WUF)
• Reading Comprehension
• MAZE
• Retell Fluency
• Early Literacy Skills
• Initial Sound Fluency (ISF)
• Letter Naming Fluency (LNF)
• Letter Sound Fluency (LSF)
• Phoneme Segmentation Fluency (PSF)
• Nonsense Word Fluency (NWF)
• Spelling
• Written Expression
• Behavior

• Math
• Math Computation
• Math Facts
• Early Numeracy
• Oral Counting
• Missing Number
• Number Identification
• Quantity Discrimination

7
Importance of RoI

• Multi-tiered model
• Progress monitoring
• Data for decision-making
• Goal setting (Shapiro, 2008)

8
Importance of RoI

• Visual inspection of slope
• Multiple interpretations
• Instructional services
• Need for explicit guidelines

9
RoI Foundations

• Deno, 1985
• Curriculum-based measurement
• General outcome measures
• Short
• Standardized
• Repeatable
• Sensitive to change

10
RoI Foundations

• Fuchs Fuchs, 1998
• Hallmark components of Response to Intervention
• Ongoing formative assessment
• Identifying non-responding students
• Treatment fidelity of instruction
• Dual discrepancy model
• One standard deviation from typically performing
  peers in level and rate

11
RoI Foundations

• Ardoin Christ, 2008
• Slope for benchmarks (3x per year)
• More growth from fall to winter than winter to
  spring
• Might be helpful to use RoI for fall to winter
• And a separate RoI for winter to spring

12
RoI Foundations

• Fuchs, Fuchs, Walz, Germann, 1993
• Typical weekly growth rates
• Needed growth
• 1.5 to 2.0 times typical slope to close gap
• Example
• Bob is below benchmark on ORF
• Typical slope is 1 wcpm per week growth
• Bob would need slope of 1.5 to 2 to close gap in
  a reasonable amount of time

13
RoI Foundations

• Deno, Fuchs, Marston, Shin, 2001
• Slope of frequently non-responsive children
  approximated slope of children already identified
  as having a specific learning disability

14
RoI Definition

• Algebraic term Slope of a line
• Vertical change over the horizontal change
• Rise over run
• m (y2 - y1) / (x2 - x1)
• Describes the steepness of a line (Gall Gall,
  2007)

15
RoI Definition

• Finding a students RoI finding the slope of a
  line
• Using two data points on that line
• Finding the line itself
• Linear regression
• Ordinary Least Squares

16
RoI Statistics

• Gall Gall, 2007
• 10 data points are a minimum requirement for a
  reliable trendline
• How does that affect the frequency of
  administering progress monitoring probes?

17
Ongoing Research

• Using RoI for instructional decisions is not a
  perfect process
• Research is currently looking to address sources
  of error
• Christ, 2006 standard error of measurement for
  slope
• Ardoin Christ, 2009 passage difficulty and
  variability
• Jenkin, Graff, Miglioretti, 2009 frequency of
  progress monitoring

18
Future Considerations

• Questions yet to be empirically answered
• What parameters of RoI indicate a lack of RtI?
• How does standard error of measurement play into
  using RoI for instructional decision making?
• How does RoI vary between standard protocol
  interventions?
• How does this apply to non-English speaking
  populations?

19
How is RoI Calculated? Which way is best?
20
Multiple Methods for Calculating Growth

• Eye ball Approach
• Last point minus First point Approach
• Split Middle Approach
• Linear Regression Approach

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
1.1 Words Per Week
25
RoI Consistency?
Eye Ball ???
Last minus First 0.75
Split Middle 0.50
Linear Regression 1.10
26
RoI Consistency?

• If we are not all using the same model to compute
  RoI, we continue to have the same problems as
  past models, where under one approach a student
  meets SLD criteria, but under a different
  approach, the student does not.
• Hypothetically, if the RoI cut-off was 0.65 or
  0.95, different approaches would come to
  different conclusions on the same student.

27
Technical Adequacy

• Without a consensus on how to compute RoI, we
  risk falling short of having technical adequacy
  within our model.

28
So, Which RoI Method is Best?
29
Literature shows that Linear Regression is Best
Practice

• Students daily test scoreswere entered into a
  computer programThe data analysis program
  generated slopes of improvement for each level
  using and Ordinary-Least Squares procedure
  (Hayes, 1973) and the line of best fit.
• This procedure has been demonstrated to represent
  CBM achievement data validly within individual
  treatment phases (Marston, 1988 Shinn, Good,
  Stein, in press Stein, 1987).
• Shinn, Gleason, Tindal, 1989

30
Growth (RoI) Research using Linear Regression

• Christ, T. J. (2006). Short-term estimates of
  growth using curriculum based measurement of oral
  reading fluency Estimating standard error of the
  slope to construct confidence intervals. School
  Psychology Review, 35, 128-133.
• Deno, S. L., Fuchs, L. S., Marston, D., Shin,
  J. (2001). Using curriculum based measurement to
  establish growth standards for students with
  learning disabilities. School Psychology Review,
  30, 507-524.
• Good, R. H. (1990). Forecasting accuracy of slope
  estimates for reading curriculum based
  measurement Empirical evidence. Behavioral
  Assessment, 12, 179-193.
• Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L.
  Germann, G. (1993). Formative evaluation of
  academic progress How much growth can we expect?
  School Psychology Review, 22, 27-48.

31
Growth (RoI) Researchusing Linear Regression

• Jenkins, J. R., Graff, J. J., Miglioretti, D.L.
  (2009). Estimating reading growth using
  intermittent CBM progress monitoring. Exceptional
  Children, 75, 151-163.
• Shinn, M. R., Gleason, M. M., Tindal, G.
  (1989). Varying the difficulty of testing
  materials Implications for curriculum-based
  measurement. The Journal of Special Education,
  23, 223-233.
• Shinn, M. R., Good, R. H., Stein, S. (1989).
  Summarizing trend in student achievement A
  comparison of methods. School Psychology Review,
  18, 356-370.

32
So, Why Are There So Many Other RoI Models?

• Ease of application
• How many of us want to calculate OLS Linear
  Regression formulas (or even remember how)?

33
An Easy and Applicable Solution
34
Get Out Your Laptops!

• Open Microsoft Excel

I love ROI
35
Graphing RoIFor Individual Students

• Programming Microsoft Excel to Graph Rate of
  Improvement
• Fall to Winter

36
Setting Up Your Spreadsheet

• In cell A1, type 3rd Grade ORF
• In cell A2, type First Semester
• In cell A3, type School Week
• In cell A4, type Benchmark
• In cell A5, type the Students Name (Swiper
  Example)

37
Labeling School Weeks

• Starting with cell B3, type numbers 1 through 18
  going across row 3 (horizontal).
• Numbers 1 through 18 represent the number of the
  school week.
• You will end with week 18 in cell S3.

38
Labeling Dates

• Note You may choose to enter the date of that
  school week across row 2 to easily identify the
  school week.

39
Entering Benchmarks(3rd Grade ORF)

• In cell B4, type 77. This is your fall benchmark.

• In cell S4, type 92. This is your winter
  benchmark.

40
Entering Student Data (Sample)

• Enter the following numbers, going across row 5,
  under corresponding week numbers.
• Week 1 41
• Week 8 62
• Week 9 63
• Week 10 75
• Week 11 64

• Week 12 80
• Week 13 83
• Week 14 83
• Week 15 56
• Week 17 104
• Week 18 74

41
CAUTION

• If a student was not assessed during a certain
  week, leave that cell blank
• Do not enter a score of Zero (0) it will be
  calculated into the trendline and interpreted as
  the student having read zero words correct per
  minute during that week.

42
Graphing the Data

• Highlight cells A4 and A5 through S4 and S5
• Follow Excel 2003 or Excel 2007 directions from
  here

43
Graphing the Data

• Excel 2003
• Across the top of your worksheet, click on
  Insert
• In that drop-down menu, click on Chart

• Excel 2007
• Click Insert
• Find the icon for Line
• Click the arrow below Line

44
Graphing the Data

• Excel 2003
• A Chart Wizard window will appear

• Excel 2007
• 6 graphics appear

45
Graphing the Data

• Excel 2003
• Choose Line
• Choose Line with markers

• Excel 2007
• Choose Line with markers

46
Graphing the Data

• Excel 2003
• Data Range tab
• Columns

• Excel 2007
• Your graph appears

47
Graphing the Data

• Excel 2003
• Chart Title
• School Week X Axis
• WPM Y Axis

• Excel 2007
• Change your labels by right clicking on the graph

48
Graphing the Data

• Excel 2003
• Choose where you want your graph

• Excel 2007
• Your graph was automatically put into your data
  spreadsheet

49
Graphing the Trendline

• Excel 2003
• Right click on any of the student data points

• Excel 2007

50
Graphing the Trendline

• Excel 2003
• Choose Linear

• Excel 2007

51
Graphing the Trendline

• Excel 2003
• Choose Custom and check box next to Display
  equation on chart

• Excel 2007

52
Graphing the Trendline

• Clicking on the equation highlights a box around
  it
• Clicking on the box allows you to move it to a
  place where you can see it better

53
Graphing the Trendline

• You can repeat the same procedure to have a
  trendline for the benchmark data points
• Suggestion label the trendline Expected ROI
• Move this equation under the first

54
Individual Student Graph
55
Individual Student Graph

• The equation indicates the slope, or rate of
  improvement.
• The number, or coefficient, before "x" is the
  average improvement, which in this case is the
  average number of words per minute per week
  gained by the student.

56
Individual Student Graph

• The rate of improvement, or trendline, is
  calculated using a linear regression, a simple
  equation of least squares.
• To add additional progress monitoring/benchmark
  scores once youve already created a graph, enter
  additional scores in Row 5 in the corresponding
  school week.

57
Individual Student Graph

• Remember to leave cells blank for the weeks that
  no score was obtained. Otherwise, the graph will
  incorporate that score into the set of data
  points and into the trendline.

58
Individual Student Graph

• The slope can change depending on which week
  (where) you put the benchmark scores on your
  chart.
• Enter benchmark scores based on when your school
  administers their benchmark assessments for the
  most accurate depiction of expected student
  progress.

59
Options for the Graph

• Resizing
• Coloring
• Data Labels

60
Programming ExcelFirst Semester

• Calculating Needed RoI
• Calculating Actual RoI Benchmark
• Calculating Actual RoI - Student

61
Calculating Needed RoI

• In cell T3, type Needed RoI
• Click on cell T5
• In the fx line (at top of sheet) type this
  formula ((S4-B5)/18)
• Then hit enter
• Your result should read 2.83333...
• This formula simply subtracts the students
  actual beginning of year (BOY) benchmark from the
  expected middle of year (MOY) benchmark, then
  dividing by 18 for the first 18 weeks (1st
  semester).

62
Calculating Actual RoI - Benchmark

• In cell U3, type Actual RoI
• Click on cell U4
• In the fx line (at top of sheet) type this
  formula SLOPE(B4S4,B3S3)
• Then hit enter
• Your result should read 0.8825...
• This formula considers 18 weeks of benchmark data
  and provides an average growth or change per week.

63
Calculating Actual RoI - Student

• Click on cell U5
• In the fx line (at top of sheet) type this
  formula SLOPE(B5S5,B3S3)
• Then hit enter
• Your result should read 2.5137...
• This formula considers 18 weeks of student data
  and provides an average growth or change per week.

64
Graphing RoIFor Individual Students

• Programming Microsoft Excel to Graph Rate of
  Improvement
• Winter to Spring

65
Setting Up Your Spreadsheet

• In cell A1, type 3rd Grade ORF
• In cell A2, type Second Semester
• In cell A3, type School Week
• In cell A4, type Benchmark
• In cell A5, type the Students Name (Swiper
  Example)

66
Labeling School Weeks

• Starting with cell B3, type numbers 1 through 18
  going across row 3 (horizontal).
• Numbers 1 through 18 represent the number of the
  school week.
• You will end with week 18 in cell S3.

67
Labeling Dates

• Note You may choose to enter the date of that
  school week across row 2 to easily identify the
  school week.

68
Entering Benchmarks(3rd Grade ORF)

• In cell B4, type 92. This is your fall benchmark.

• In cell S4, type 110. This is your winter
  benchmark.

69
Entering Student Data (Sample)

• Enter the following numbers, going across row 5,
  under corresponding week numbers.
• Week 1 74
• Week 3 85
• Week 4 89
• Week 5 69
• Week 6 85

• Week 7 96
• Week 8 90
• Week 9 84
• Week 10 106
• Week 11 94
• Week 15 100

70
CAUTION

• If a student was not assessed during a certain
  week, leave that cell blank
• Do not enter a score of Zero (0) it will be
  calculated into the trendline and interpreted as
  the student having read zero words correct per
  minute during that week.

71
Graphing the Data

• Highlight cells A4 and A5 through S4 and S5
• Follow Excel 2003 or Excel 2007 directions from
  here

72
Graphing the Data

• Excel 2003
• Across the top of your worksheet, click on
  Insert
• In that drop-down menu, click on Chart

• Excel 2007
• Click Insert
• Find the icon for Line
• Click the arrow below Line

73
Graphing the Data

• Excel 2003
• A Chart Wizard window will appear

• Excel 2007
• 6 graphics appear

74
Graphing the Data

• Excel 2003
• Choose Line
• Choose Line with markers

• Excel 2007
• Choose Line with markers

75
Graphing the Data

• Excel 2003
• Data Range tab
• Columns

• Excel 2007
• Your graph appears

76
Graphing the Data

• Excel 2003
• Chart Title
• School Week X Axis
• WPM Y Axis

• Excel 2007
• Change your labels by right clicking on the graph

77
Graphing the Data

• Excel 2003
• Choose where you want your graph

• Excel 2007
• Your graph was automatically put into your data
  spreadsheet

78
Graphing the Trendline

• Excel 2003
• Right click on any of the student data points

• Excel 2007

79
Graphing the Trendline

• Excel 2003
• Choose Linear

• Excel 2007

80
Graphing the Trendline

• Excel 2003
• Choose Custom and check box next to Display
  equation on chart

• Excel 2007

81
Graphing the Trendline

• Clicking on the equation highlights a box around
  it
• Clicking on the box allows you to move it to a
  place where you can see it better

82
Graphing the Trendline

• You can repeat the same procedure to have a
  trendline for the benchmark data points
• Suggestion label the trendline Expected ROI
• Move this equation under the first

83
Individual Student Graph
84
Individual Student Graph

• The equation indicates the slope, or rate of
  improvement.
• The number, or coefficient, before "x" is the
  average improvement, which in this case is the
  average number of words per minute per week
  gained by the student.

85
Individual Student Graph

• The rate of improvement, or trendline, is
  calculated using a linear regression, a simple
  equation of least squares.
• To add additional progress monitoring/benchmark
  scores once youve already created a graph, enter
  additional scores in Row 5 in the corresponding
  school week.

86
Individual Student Graph

• Remember to leave cells blank for the weeks that
  no score was obtained. Otherwise, the graph will
  incorporate that score into the set of data
  points and into the trendline.

87
Individual Student Graph

• The slope can change depending on which week
  (where) you put the benchmark scores on your
  chart.
• Enter benchmark scores based on when your school
  administers their benchmark assessments for the
  most accurate depiction of expected student
  progress.

88
Options for the Graph

• Resizing
• Coloring
• Data Labels

89
Programming ExcelSecond Semester

• Calculating Needed RoI
• Calculating Actual RoI Benchmark
• Calculating Actual RoI - Student

90
Calculating Needed RoI

• In cell T3, type Needed RoI
• Click on cell T5
• In the fx line (at top of sheet) type this
  formula ((S4-B5)/18)
• Then hit enter
• Your result should read 2
• This formula simply subtracts the students
  actual middle of year (MOY) benchmark from the
  expected end of year (EOY) benchmark, then
  dividing by 18 for the first 18 weeks (1st
  semester).

91
Calculating Actual RoI - Benchmark

• In cell U3, type Actual RoI
• Click on cell U4
• In the fx line (at top of sheet) type this
  formula SLOPE(B4S4,B3S3)
• Then hit enter
• Your result should read 1.06
• This formula considers 18 weeks of benchmark data
  and provides an average growth or change per week.

92
Calculating Actual RoI - Student

• Click on cell U5
• In the fx line (at top of sheet) type this
  formula SLOPE(B5S5,B3S3)
• Then hit enter
• Your result should read 1.89
• This formula considers 18 weeks of student data
  and provides an average growth or change per week.

93
ROI as a Decision Tool

• within a Problem-Solving Model

94
Steps

1. Gather the data
2. Ground the data set goals
3. Interpret the data
4. Figure out how to fit Best Practice into Public
   Education

95
Step 1 Gather Data

• Universal Screening
• Progress Monitoring

96
Common Screenings in PA

• DIBELS
• AIMSweb
• MBSP
• 4Sight
• PSSA

97
Validated Progress Monitoring Tools

• DIBELS
• AIMSweb
• MBSP
• www.studentprogress.org

98
Step 2 Ground the Data

• 1) To what will we compare our student growth
  data?
• 2) How will we set goals?

99
Multiple Ways to Look at Growth

• Needed Growth
• Expected Growth Percent of Expected Growth
• Fuchs et. al. (1993) Table of Realistic and
  Ambitious Growth
• Growth Toward Individual Goal
• Best Practices in Setting Progress Monitoring
  Goals for Academic Skill Improvement (Shapiro,
  2008)

100
Needed Growth

• Difference between students BOY (or MOY) score
  and benchmark score at MOY (or EOY).
• Example MOY ORF 10, EOY benchmark is 40, 18
  weeks of instruction (40-10/181.67). Student
  must gain 1.67 wcpm per week to make EOY
  benchmark.

101
Expected Growth

• Difference between two benchmarks.
• Example MOY benchmark is 20, EOY benchmark is
  40, expected growth (40-20)/18 weeks of
  instruction 1.11 wcpm per week.

102
Looking at Percent of Expected Growth
Tier I Tier II Tier III
Greater than 150
Between 110 150 Possible LD
Between 95 110 Likely LD
Between 80 95 May Need More May Need More Likely LD
Below 80 Needs More Needs More Likely LD
103
Oral Reading Fluency Adequate Response Table
Realistic Growth Ambitious Growth
1st 2.0 3.0
2nd 1.5 2.0
3rd 1.0 1.5
4th 0.9 1.1
5th 0.5 0.8
104
Digit Fluency Adequate Response Table
Realistic Growth Ambitious Growth
1st 0.3 0.5
2nd 0.3 0.5
3rd 0.3 0.5
4th 0.75 1.2
5th 0.75 1.2
105
From Where Should Benchmarks/Criteria Come?

• Appears to be a theoretical convergence on use of
  local criteria (what scores do our students need
  to have a high probability of proficiency?) when
  possible.

106
Steps to Develop Local Criteria

• Not enough time today!
• See us in State College in the fall, or
• Check out our website later this summer.

107
If Local Criteria are Not an Option

• Use norms that accompany the measure (DIBELS,
  AIMSweb, etc.).
• Use national norms.

108
Making Decisions Best Practice

• Research has yet to establish a blue print for
  grounding student RoI data.
• At this point, teams should consider multiple
  comparisons when planning and making decisions.

109
Making Decisions Lessons From the Field

• When tracking on grade level, consider an RoI
  that is 100 of expected growth as a minimum
  requirement, consider an RoI that is at or above
  the needed as optimal.
• So, 100 of expected and on par with needed
  become the limits of the range within a student
  should be achieving.

110
(No Transcript)
111
Research Support

• RoI by half year vs. whole year (Curvilinear
  Growth).
• Expected growth as mediated by initial level.

112
Example of Curvilinear Growth
BOY to MOY 1.60 MOY to BOY 1.19 BOY to EOY
1.35
113
Ardoin Christ, 2008

• Slope for benchmarks (3x per year)
• More growth from fall to winter than winter to
  spring

114
Christ, Yeo, Silberglitt, in press

• Growth across benchmarks (3X per year)
• More growth from fall to winter than winter to
  spring
• Disaggregated special education population

115
Graney, Missall, Martinez, 2009

• Growth across benchmarks (3X per year)
• More growth from winter to spring than fall to
  winter with R-CBM.

116
Fien, Park, Smith, Baker, 2010

• Investigated relationship b/w NWF gains and
  ORF/Comprehension
• Found greater NWF gains in fall than in spring.

117
DIBELS ORF Change in Criteria
Fall to Winter Winter to Spring
2nd 24 22
3rd 15 18
4th 13 13
5th 11 9
6th 11 5
118
AIMSweb Norms
Based on 50th Percentile Fall to Winter Winter to Spring
1st 18 31
2nd 25 17
3rd 22 15
4th 16 13
5th 17 15
6th 13 12
119
Speculation as to why Differences in RoI within
the Year

• Relax instruction after high stakes testing in
  March/April a PSSA effect.
• Depressed BOY benchmark scores due to summer
  break a rebound effect (Clemens).
• Instructional variables could explain differences
  in Graney (2009) and Ardoin (2008) Christ (in
  press) results (Silberglitt).
• Variability within progress monitoring probes
  (Ardoin Christ, 2008) (Lent).

120
RoI Research

• Growth Mediated by Level

121
Fien, Park, Smith, Baker, 2010

• Nonsense Word Fluency
• Different growth rates depending on beginning
  level

122
Clemens, 2010

• Investigated NWF and WIF
• NWF slope validity increased as initial skills
  were lower, but relationships with outcomes
  similar to WIF

123
Silberglitt Hintze, 2007

• R-CBM
• Differences in growth rates depending on level
• Lowest and highest deciles had least amount of
  growth

124
Good et. al., 2010
Growth Rate as Function of Level at BOY (2nd Grade) Growth Rate as Function of Level at BOY (2nd Grade) Growth Rate as Function of Level at BOY (2nd Grade) Growth Rate as Function of Level at BOY (2nd Grade) Growth Rate as Function of Level at BOY (2nd Grade) Growth Rate as Function of Level at BOY (2nd Grade)
20th 40th 60th 80th
Intensive 0 to 5 0.11 0.33 0.56 0.98
Intensive 6 to 15 0.40 0.70 1.05 1.53
Intensive 16 to 25 0.95 1.43 1.78 2.20
Strategic 26 to 34 1.30 1.73 2.06 2.43
Strategic 35 to 43 1.50 1.83 2.11 2.50
125
Conclusions

• Appear to be different RoI within the school
  year.
• Compute RoI goals by half-year (Fall to Winter,
  Winter to Spring).
• Actual or Expected RoI appears to differ
  depending on the level at which a student
  originally scores, which could have goal setting
  ramifications.

126
Step 3 Interpreting Growth
127
What do we do when we do not get the growth we
want?

• When to make a change in instruction and
  intervention?
• When to consider SLD?

128
When to make a change in instruction and
intervention?

• Enough data points (6 to 10)?
• Less than 100 of expected growth.
• Not on track to make benchmark (needed growth).
• Not on track to reach individual goal.

129
When to consider SLD?

• Continued inadequate response despite
• Fidelity with Tier I instruction and Tier II/III
  intervention.
• Multiple attempts at intervention.
• Individualized Problem-Solving approach.

130
Three Levels of Examples

• Whole Class
• Small Group
• Individual Student
• - Academic Data
• - Behavior Data

131
Whole Class Example
132
3rd Grade Math Whole Class

• Whos responding?
• Effective math instruction?
• Who needs more?

• N19
• 4 gt 100 growth
• 15 lt 100 growth
• 9 w/ negative growth

133
Small Group Example
134
Intervention Group

• Intervention working for how many?
• Can we assume fidelity of intervention based on
  results?
• Who needs more?

135
Individual Kid Example
136
Individual Kid

• Making growth?
• How much (65 of expected growth).
• Atypical growth across the year (last 3 data
  points).
• Continue? Make a change? Need more data?

137
RoI and Behavior?

138
(No Transcript)
139
Step 4 Figure out how to fit Best Practice into
Public Education
140
Things to Consider

• Who is At-Risk and needs progress monitoring?
• Who will collect, score, enter the data?
• Who will monitor student growth, when, and how
  often?
• What changes should be made to instruction
  intervention?
• What about monitoring off of grade level?

141
Who is At-Risk and needs progress monitoring?

• Below level on universal screening

Entering 4th Grade Example Entering 4th Grade Example Entering 4th Grade Example Entering 4th Grade Example Entering 4th Grade Example
DORF (110) ISIP TRWM (55) 4Sight (1235) PSSA (1235)
Student A 115 58 1255 1232
Student B 85 48 1216 1126
Student C 72 35 1056 1048
142
Who will collect, score, and enter the data?

• Using MBSP for math, teachers can administer
  probes to whole class.
• DORF probes must be administered one-on-one, and
  creativity pays off (train and use art, music,
  library, etc. specialists).
• Schedule for progress monitoring math and reading
  every-other week.

143
Week 1 Week 1 Week 2 Week 2
Reading Math Reading Math
1st X X
2nd X X
3rd X X
4th X X
5th X X
144
Who will monitor student growth, when, and how
often?

• Best Practices in Data-Analysis Teaming
  (Kovaleski Pedersen, 2008)
• Chambersburg Area School District Elementary
  Response to Intervention Manual (McCrea et. al.,
  2008)
• Derry Township School District Response to
  Intervention Model (http//www.hershey.k12.pa.us/5
  6039310111408/lib/56039310111408/_files/Microsoft_
  Word_-_Response_to_Intervention_Overview_of_Hershe
  y_Elementary_Model.pdf)

145
What changes should be made to instruction
intervention?

• Ensure treatment fidelity!!!!!!!!
• Increase instructional time (active and engaged)
• Decrease group size
• Gather additional, diagnostic, information
• Change the intervention

146
When Instructional Level is Not the Same as Grade
Level

• Understand needed and expected RoI within broader
  context
• Needed growth will only get student to next level
  by next benchmark (as opposed to on level).
• 100 of expected growth may not be an acceptable
  minimum (not enough growth b/c level is so low).

147
Grounding RoI When Monitoring Off of Grade Level
Three Options

• Best Practices in Setting Progress Monitoring
  Goals for Academic Skill Improvement (Shapiro,
  2008).
• Shinn approach as detailed in AIMSweb training
  workshop on Progress Monitoring.
• Tigard-Tualatin SD Chart 150 instead of 100 as
  minimum RoI requirement???

148
Questions? Comments!
149
Resources

• www.interventioncentral.com
• www.aimsweb.com
• http//dibels.uoregon.edu
• www.nasponline.org

150
Resources

• www.fcrr.org
• Florida Center for Reading Research
• http//ies.ed.gov/ncee/wwc//
• What Works Clearinghouse
• http//www.rti4success.org
• National Center on RtI

151
Flinn McCreas RoI Web Site

• http//sites.google.com/site/rateofimprovement/
• Download powerpoints, handouts, Excel graphs,
  charts, articles, etc.
• Caitlin Flinn
• c.s.flinn_at_iup.edu
• Andrew McCrea
• mccreand_at_chambersburg.k12.pa.us

152
References

• Ardoin, S. P., Christ, T. J. (2009).
  Curriculum-based measurement of oral reading
  Standard errors associated with progress
  monitoring outcomes from DIBELS, AIMSweb, and an
  experimental passage set. School Psychology
  Review, 38(2), 266-283.
• Ardoin, S. P. Christ, T. J. (2008). Evaluating
  curriculum-based measurement slope estimates
  using triannual universal screenings. School
  Psychology Review, 37(1), 109-125.

153
References

• Christ, T. J. (2006). Short-term estimates of
  growth using curriculum-based measurement of oral
  reading fluency Estimating standard error of the
  slope to construct confidence intervals. School
  Psychology Review, 35(1), 128-133.
• Deno, S. L. (1985). Curriculum-based measurement
  The emerging alternative. Exceptional Children,
  52, 219-232.

154
References

• Deno, S. L., Fuchs, L.S., Marston, D., Shin, J.
  (2001). Using curriculum-based measurement to
  establish growth standards for students with
  learning disabilities. School Psychology Review,
  30, 507-524.
• Flinn, C. S. (2008). Graphing rate of improvement
  for individual students. InSight, 28(3), 10-12.

155
References

• Fuchs, L. S., Fuchs, D. (1998). Treatment
  validity A unifying concept for
  reconceptualizing the identification of learning
  disabilities. Learning Disabilities Research and
  Practice, 13, 204-219.
• Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz,
  L., Germann, G. (1993). Formative evaluation of
  academic progress How much growth can we expect?
  School Psychology Review, 22, 27-48.

156
References

• Gall, M.D., Gall, J.P. (2007). Educational
  research An introduction (8th ed.). New York
  Pearson.
• Jenkins, J. R., Graff, J. J., Miglioretti, D.L.
  (2009). Estimating reading growth using
  intermittent CBM progress monitoring. Exceptional
  Children, 75, 151-163.

157
References

• Karwowski, W. (2006). International encyclopedia
  of ergonomics and human factors. Boca Raton, FL
  Taylor Francis Group, LLC.
• Shapiro, E. S. (2008). Best practices in setting
  progress monitoring goals for academic skill
  improvement. In A. Thomas and J. Grimes (Eds.),
  Best practices in school psychology V (Vol. 2,
  pp. 141-157). Bethesda, MD National Association
  of School Psychologists.

158
References

• Vogel, D. R., Dickson, G. W., Lehman, J. A.
  (1990). Persuasion and the role of visual
  presentation support. The UM/3M study. In M.
  Antonoff (Ed.), Presentations that persuade.
  Personal Computing, 14.