Response to Intervention: Using Data to Enhance Outcomes for all Students - PowerPoint PPT Presentation

1 / 148
About This Presentation
Title:

Response to Intervention: Using Data to Enhance Outcomes for all Students

Description:

Amanda VanDerHeyden Education Research and Consulting, Inc. Objectives Today Overview of RTI, RTI decision making, and expected outcomes Specific How-To for RTI ... – PowerPoint PPT presentation

Number of Views:259
Avg rating:3.0/5.0
Slides: 149
Provided by: AmandaMVa6
Category:

less

Transcript and Presenter's Notes

Title: Response to Intervention: Using Data to Enhance Outcomes for all Students


1
Response to Intervention Using Data to Enhance
Outcomes for all Students
  • Amanda VanDerHeyden
  • Education Research and Consulting, Inc.

2
Objectives Today
  • Overview of RTI, RTI decision making, and
    expected outcomes
  • Specific How-To for RTI
  • Interpreting Assessment Data to determine need
    for Tiers 1, 2, and 3 Interventions
  • Selection and implementation of Tier 2
    Interventions
  • Selection and implementation of Tier 3
    Interventions
  • Implementing intervention for sustenance and
    system change

3
16 x 3 48 hours
4
What is RTI?
  • A science of decision making and way of thinking
    about how educational resources can be allocated
    (or reallocated) to best help all children learn
  • Major premium on child outcomes

5
RTI is Not
  • A program, a curriculum, an intervention, a
    particular model

6
Data allow us to
  • Provide faster, more effective services for ALL
    children
  • Work smarter not harder, better utilize the
    talents of the school psychologist and
    school-based assessment and intervention teams.
  • Make implementation SIMPLE and EASY for teachers
    (low cost, few errors)
  • Prevent diagnosis

7
Early Screening Identifies Children At Risk of
Reading Difficulty
J
5
4
Low Risk on Early Screening
Reading grade level
3
2
At Risk on Early Screening
1
1 2 3 4
Grade level corresponding to age
This Slide from Reading First Experts
From Reading First
8
Early Intervention Changes Reading Outcomes
J
5.2
5
4
Low Risk on Early Screening
Reading grade level
3
2.5
2
At Risk on Early Screening
1
1 2 3 4
Grade level corresponding to age
This Slide from Reading First Experts
From Reading First
9
Evolution
  • Wait to Fail
  • Lets provide services early!
  • Costly sp ed programs not improving learning
  • Lets shift resources to provide services in less
    restrictive setting!
  • Increasing numbers of children struggling in
    general ed
  • Lets provide help in general education!
  • Traditional measures are de-contextualized and
    the constructs are problematic
  • Lets help children who struggle academically by
    measuring performance in response to certain
    intervention strategies and then deliver what
    works!

10
Rationale for System Change
200-300 increase in SLD
  • Level and Rate of Performance
  • Return to General Education
  • Lack of Certified Teachers
  • No demonstrated instructional techniques that
    differentially benefit SLD
  • Drop-out
  • Disproportionate Representation by Ethnicity

11
History of RTI
  • Effective Instruction Lit, CBA/M Lit
  • Lab-Quality Intervention Programs
  • Progress Monitoring Data and Problem-Solving
    Models
  • Reading First data

12
Consistent with
  • NCLB
  • Reauthorized IDEA
  • Recommendations of panel reports on Minority
    students in special education, National Reading
    Panel, Science and Math Initiative

13
Impetus
  • Faster, more effective services for ALL children
  • Work smarter not harder, better utilize the
    talents of the school psychologist and
    school-based assessment and intervention teams.
  • Make implementation SIMPLE and EASY for teachers
    (low cost, few errors)

14
Why RTI?
  • Viable alternative to traditional diagnosis of
    high-incidence disabilities, particularly
    Learning Disability (LD)
  • Reauthorized IDEA guidelines for identifying LD
    state that
  • a) A severe discrepancy between achievement and
    intellectual ability shall not be required
  • b) A response to intervention (RTI) may be
    considered

15
Why RTI?
  • RTI can address the problem of disproportionate
    identification of children with LD by race and
    gender
  • The utility of curriculum-based measures (CBM)
    for
  • Identifying children not likely to benefit from
    the general education curriculum without
    assistance,
  • Predicting important long-term outcomes
  • Tracking individual student growth and informing
    instructional programming changes has been
    established

16
Considerations
  • There has been some consensus concerning the need
    for change however, there has not been consensus
    on how this change can best be achieved.
  • Whereas RTI has considerable promise as a tool
    within special and general education, it is a
    vulnerable construct if misapplied.

17
Improved Treatment Validity
  • Direct link to treatment or consequential
    validity
  • Efforts are focused to
  • Properly articulate a concern
  • Develop targeted intervention to resolve the
    concern
  • Collect information to determine whether or not
    the concern has been adequately addressed or
    whether different solution efforts need to be
    implemented
  • This approach changes the goal of assessment from
    what some have described as admiration of the
    problem to problem-solving.

18
Contextualized Decision Making
  • RTI emphasizes the pre-referral conditions (child
    and environment) and this context becomes part of
    the decision-making equation.
  • Allows practitioners to quantify
  • The state of instructional affairs in the childs
    regular education environment
  • Potential learning given optimal instructional
    conditions
  • RTI may enable improvement in general education
    programming leading to children receiving
    assistance in a more efficient manner.

19
Improved Identification Accuracy for LD
  • Under RTI models intervention becomes a
    specified, operationalized variable, thus false
    positive identification errors should be reduced
    dramatically.
  • Removing the current reliance on teacher
    identification and requiring direct measures of
    child performance in context will enhance
    identification accuracy.

20
More Effective Intervention
  • RTI is likely to facilitate less restrictive
    interventions and placements for children.
  • RTI allows school psychologists to bring their
    expertise to bear on assessment strategies at the
    classroom level and assist teachers to use data
    formatively to enhance their instructional
    programming.

21
  • Possible Challenges of RTI

22
Decision-making Criteria
  • Must be operationalized and validated through
    research
  • The purpose of RTI will be critical to
    determining how implementation should proceed

23
New Challenges for Teams
  • Effective intervention delivery will depend on
    relevant intervention variables
  • To be effective the intervention must have been
  • Properly identified
  • Implemented with integrity and with sufficient
    frequency, intensity, and duration

24
STEEP Model
  • Screening to Enhance Educational Progress

25
Tier 1 Math Screening
  • Math Probe
  • Group administered.
  • Materials Worksheet consisting of a series of
    problems sampling the target skill(s) (e.g., sums
    to 5, double digit multiplication with
    regrouping).
  • Timing 2 minutes
  • Information obtained digits correct in two
    minutes.

26
Math Probe Example
  • Total Digits 38
  • Errors 5
  • Digits Correct 33

27
Tier 1 Writing Screening
  • Writing Probe
  • Group administered.
  • Materials story starter (e.g., If I had a
    million dollars) printed at the top of a blank
    page.
  • Timing 1 minute to think, 3 minutes to write.
  • Scoring words written or correct word sequences
    in three minutes.

28
Writing Example
29
Tier 1 Reading Screening
  • Reading Probe
  • Individually administered
  • Materials A content-controlled reading passage.
  • Procedure The student reads aloud as the teacher
    listens and records errors.
  • Timing 1 minute
  • Information obtained words read correctly in one
    minute.

30
CBM Reading Sample Scoring
  • TRW63
  • Errors6
  • CRW58

31
Class-wide Screening
32
Feedback to Teachers
33
Tier 2 Class-wide Intervention
34
No Class-wide Problem Detected
35
Tier 2 Cant Do/Wont Do Assessment
3-7 minutes per child
  • Cant Do/Wont Do
  • Individually-administered
  • Materials
  • Academic material that student performed poorly
    during class assessment.
  • Treasure chest plastic box filled with tangible
    items.

36
Cant Do/Wont Do Assessment
37
Decision Rule Following Cant Do/Wont Do
Assessment
38
Tier 3 Individual Intervention
39
Response to Intervention
Before Intervention
During Intervention
Correct
Avg. for his Class
Each Dot is one Day of Intervention
Intervention Sessions
Intervention in Reading
40
Response to Intervention
Before Intervention
During Intervention


Correct
Avg. for his Class
41
Vehicle for System ChangeSystem-wide Math
Problem
Instructional range
Frustrational range
Each bar is a students performance
42
Re-screening Indicates No Systemic Problem
Fourth Grade
43
Rest of Grade at Standard
A
B
C
D
E
F
Classroom
44
Spring 2003 Classroom F
F
45
Teacher moved to lower grade in Fall 2003
46
Class-wide Intervention
Teacher F Mult 0-12
120
100
80
Digits Correct Two Minutes
60
40
20
0
11/7/2003
10/24/2003
10/31/2003
11/14/2003
11/18/2003
Weeks
47
Increased Difficulty- Intervention Continues
48
Mixed Mult/Div/Fractions Probe Classroom F
49
Growth Obtained
actual growth
aimline
50
Effect on High-Stakes Scores
VanDerHeyden, in prep
51
Effect on High-Stakes Scores
VanDerHeyden, in prep
52
District-wide Implementation Data
  • Vail Unified School District
  • www.vail.k12.az.us
  • Three years, system-wide implementation of STEEP
    grades 1-8

53
System Outcomes
  • Referrals reduced greater than half
  • who qualify from 50 stable baseline over three
    years to nearly 100
  • SLD down from 6 of children in district in
    2001-2002 (with baseline upward trend) to 3.5 in
    2003-2004 school year
  • Corresponding gains on high-stakes tests
    (VanDerHeyden Burns, 2005)
  • Intervention successful for about 95 to 98 of
    children screened

VanDerHeyden, Witt, Gilbertson, 2007
54
Cost Reduction
VanDerHeyden, Witt, Gilbertson, 2007
55
Findings
  • Number of Evaluations dramatically reduced 70
    at highest referral school
  • Diverse settings, psychologists of diverse
    backgrounds and no prior experience with CBM or
    functional academic assessment
  • Percentage qualify increased at 4 of 5 schools
  • Disproportionate representation of males
    positively affected
  • Number of children placed dramatically reduced

VanDerHeyden, Witt, Gilbertson, 2007
56
Team Decision-Making Agreement
RTI and Evaluated RTI- and Did Not Evaluate
2003-2004 (3 schools) 100 41
2004-2005 (5 schools) 100 87
VanDerHeyden, Witt, Gilbertson, 2007
57
Team Decision-Making
VanDerHeyden, Witt, Gilbertson, 2007
58
Fall to Spring Reading Growth
VanDerHeyden Witt, 2005
59
What Proportion of Ethnicity Represented Before
and After Intervention in Risk Category?
VanDerHeyden Witt, 2005
60
Criterion
Criterion -
13 9
4 75
PVS
PVS -
.76/.116.9 children with learning problem about
7 times more likely to have a failed RTI than
children without a learning problem .24/.89.27
children without a learning problem are 3.7 times
more likely to have a successful RTI than those
children with a learning problem
Sens- .76 Spec- .89 PPP- .59 NPP- .95
LR- sens/(1-spec) - LR- (1-sens)/spec
61
Criterion
Criterion -
16 39
1 45
Probe
Probe -
.94/.462.0 children with learning problem about
2 times more likely to perform below screening
benchmark than children without a learning
problem .06/.54.11 children without a learning
problem are 9 times more likely to have a above
criterion screening score than those children
with a learning problem
Sens- .94 Spec- .54 PPP- .29 NPP- .98
62
Identification Accuracy
  • High-achieving classrooms (lt20)
  • Procedures paired with RTI criterion were more
    accurate than other commonly used screening
    devices
  • Low-achieving classrooms (gt50)
  • Procedures paired with RTI criterion were more
    accurate than other commonly used screening
    devices

VanDerHeyden Witt, 2005
63
Questions
  • Is there a classwide problem?
  • Is there a gradewide problem?
  • Whats the most efficient way to deliver
    intervention?

64
Screening tells you
  • How is the core instruction working?
  • What problems might exist that could be
    addressed?
  • Most bang-for-the-buck activity
  • Next most high-yield activity is classwide
    intervention at Tier 2.

65
Screening Guidelines
  • Efforts at Tier 1 pay off with fewer children
    needing individual intervention
  • 3 times per year, single probe
  • Use small team of trained coaches
  • Prepare all needed materials in a packet for each
    teacher
  • Score and return within 1 week on graph
  • Use data to generate aimlines, can be used to set
    benchmarks

66
Here is 3rd grade at Cottonwood. Each circle
corresponds to a students score on a reading CBM
probe in March and the AIMS reading score the
same month. So the circle near the blue lines is
a child who read 158 wc/min and scored 486 on the
AIMS Reading. The diagonal line represents the
best fit line or the line closest to all the
circles. This shows there is a strong positive
correlation between CBM and AIMS reading scores
with this group.
67
How to Set a Benchmark
This is words read correctly per minute. You move
this line up and down to catch as many of those
who will not pass as possible.
431 pass for AIMS. This line does not move
68
Setting 95 wc/min as the pass standard for CBM
These are children who were predicted to pass
AIMS based on CBM and did pass. Hits
These are the children predicted to pass AIMS who
actually failed. False Negative Errors. The
worst kind of error.
These are the children predicted to fail AIMS who
actually passed. False Positive Errors.
These are the children who were predicted to fail
AIMS based on CBM who did fail. Hits
69
Moving the horizontal line up will catch two
more cases who failed the AIMS, but will result
in many more false positive errors.
Thus, 95 wc/min at 3rd grade at Cottonwood is a
good standard that will tell you which children
are likely to fail the AIMS reading section.
70
ROC printout
71
Mastery Model Measurement (CBA)
Letter naming fluency
Isolated sound fluency
Beginning sound fluency
Ending sound fluency
72
General Outcome Measurement (CBM)
Beginning sound fluency
Letter naming fluency
Isolated sound fluency
Ending sound fluency
Words read correctly per minute
73
Tracking Year-Long Growth
mastery
aimline
instructional
74
Digits Correct Two Minutes
12
1
Weeks
75
Any Curriculum Area
1-5
1-5
5-10
5-10
Students
80-90
80-90
Dave Tilly, 2005
76
Weighing a cow doesnt make it fatter.
77
Any Curriculum Area
1-5
1-5
5-10
5-10
Students
80-90
80-90
Dave Tilly, 2005
78
Class-wide Intervention
  • Use pair-peered practice (classwide peer
    tutoring, PALS)
  • Model, Guided Practice, Independent timed
    practice with delayed error correction, reward
    contingency

79
Instructional Hierarchy
Generalization
Fluency
Acquisition
80
(No Transcript)
81
Reading classwide intervention
82
Select a Few Good Interventions to Keep it Simple
Classwide Individual
Math Flash card Practice Cover copy compare Cue Cards Highlighted errors
Reading Listening Preview Repeated Readings Error Correction Key Words
83
(No Transcript)
84
(No Transcript)
85
Intervention Plan- 15 Min per Day
  • Protocol-based classwide peer tutoring,
    randomized integrity checks by direct observation
  • Model, Guide Practice, Independent Timed Practice
    with delayed error correction
  • Group performance contingency
  • Teachers encouraged to
  • Scan papers for high error rates
  • Do 5-min re-teach for those with high-error rates
  • Provide applied practice using mastery-level
    computational skill

86
(No Transcript)
87
Sample Sequence
88
Intervention Plan
  • Class Median reaches mastery range for skill,
    next skill is introduced
  • Following promising results at one site in
    2002-2003, lead to implementation district-wide
    grades 1-8 for all children by 2004-2005.

89
Class-wide Math Intervention
90
With teacher support
  • Consider time, resources, materials
  • Remove skill barriers with
  • classroom training for students
  • classroom coaching for teachers
  • Remove implementation barriers after use new
    steps
  • follow-up supportive meetings to problem solve.
  • frequent acknowledgment of a teachers efforts

91
Training Package
Tell Rational Step by step
protocol Show Model Do Train
students Implement with guided practice
Implement independently with support
92
  • 88 of interventions are not used
  • without support
  • Decisions do not always correspond to data
    (someone must check)

93

Tier 2 Intervention Effects
94
Class 1 at Screening
95
Class 1 Following 10 Days Intervention
96
Class 1 Following 15 Days Intervention
97
Class 2 at Screening
98
Class 2 Following 5 Days Intervention
99
Class 2 Following 10 days Intervention
100
Class 3 at Screening
101
Class 3 Following 5 days Intervention
102
Following 10 Days Intervention
103
Tier 1 Screening Indicates Class-wide Problem
104
Tier 2 Class-wide Intervention
Teacher F Mult 0-12
120
100
80
Digits Correct Two Minutes
60
40
20
0
11/7/2003
10/24/2003
10/31/2003
11/14/2003
11/18/2003
Weeks
105
Increased Difficulty- Intervention Continues
106
Contextually-Relevant Comparisons and Use of
Trend Data
107
5th Grade Math Intervention
108
Pre-post changes to performance detected by CBM
Instructional range
Frustrational range
Each bar is a students performance
109
Fourth Grade
110
Effect on SAT-9 Performance
111
Effect on CBM Scores
112
Computation Gains Generalized to High Stakes
TestImprovements (Gains within Multiple
Baselineshown as pre-post data)
113
Gains within Multiple Baseline (shown as
pre-post data)
114
Identification Accuracy
CBA RTI Criterion ITBS WJ-R
STEEP Sensitivity .76 1 .58
STEEP Specificity .89 .99 .77
STEEP Positive Predictive Power .59 .67 .44
STEEP Negative Predictive Power .95 1 .86
Teacher Referral Sensitivity .46 .33 .42
Teacher Referral Specificity .69 .94 .85
Teacher Referral Positive Predictive Power .19 .17 .45
Teacher Referral Negative Predictive Power .89 .97 .83
VanDerHeyden, et al., 2003
115
Percent Identified at each Tier
Identified
CBM (Classwide Assessment) 55 (15)
CBM Reward (Performance/skill Deficit Assessment) 40 (11)
CBM Reward Instruction (STEEP ) 22 (6)
Teacher Referral 32 (19)
CIBS-R 64 (18)
DRA 17 (9)
RTI Criterion Assessment 17 (5)
WJ-R 12
ITBS deficit 3 (4)
VanDerHeyden, et al., 2003
116
Any Curriculum Area
1-5
1-5
5-10
5-10
Students
80-90
80-90
Dave Tilly, 2005
117
Tier 3
  • Assessment Data
  • Instructional level performance
  • Error analysis (high errors, low errors, pattern)
  • Effect of incentives, practice, easier task
  • Verify intervention effect
  • Same implementation support as Tier 2
  • Instructional-level materials Criterion-level
    materials

118
Assessment Data
119
Assessment Data
120
Tier 3
  • Implement for 5-15 consecutive sessions with 100
    integrity
  • Link to referral decision
  • Weekly graphs to teacher and weekly
    generalization probes outside of classroom,
    supply new materials
  • Troubleshoot implementation weekly

121
Lessons Learned
  • Most individual interventions for reading
  • Standard protocols with slight modifications are
    best
  • Most interventions are successful (should be
    successful)
  • Generalization must be attended to
  • Team will not follow data without support and
    training to do so
  • Coordinate intervention start times with
    principal and stagger start dates (10-15 at a
    time plus Tier 2s).
  • Organize master schedule for data collection and
    Tier times

122
Tier 3 Intervention
  • gt5 of children screened (total population) IF
    solid Tier 1
  • Possibly as low as 2 IF solid Tier 1 and Tier 2
  • About 1-2 failed RTI 10 of most at-risk

123
Successful RTI
124
2004- 3rd Grade
125
Successful RTI
126
Successful Writing
127
Successful RTI
128
Measurement Error
129
Successful Math
130
Unsuccessful Math
131
Tips for Effective Implementation
132
Our Recipe for Intervention Success
  • PREPARE
  • Identify and Use standard protocols for
    intervention
  • Develop all needed materials
  • Develop packets or put on a central web site
  • Determine graphing program

133
Our Recipe for Intervention Success
  • TRAIN
  • Explain
  • Watch the teacher do it with the actual child
    before you leave
  • Call or meet teacher after first day to problem
    solve

134
Our Recipe for Intervention Success
  • COLLECT DATA AND SUPPORT
  • Each week, graph intervention performance and do
    a generalization check with the child.
  • Graphed feedback to teachers with generalization
    checks for individual intervention once per week
  • Response-dependent performance feedback to
    sustain implementation accuracy
  • Monthly CBM to track growth and enhance existing
    Tier 1 Programs or advise new Tier 1
  • Data to principal weekly. Summarize effects and
    integrity of procedures.

135
Tracking Title Progress
136
Our Recipe for Intervention Success
  • USE DATA TO MAKE DECISIONS
  • RTI successful if child performs criterion-level
    probe (from screening) in the instructional
    range. RTI unsuccessful if 15 consecutive
    intervention sessions and criterion probe is not
    in the instructional range.
  • Increase task difficulty for intervention if
    child scores at mastery on task during
    intervention sessions

137
Infrastructure for Implementation
  • Grade-level planning periods can be utilized
  • Special education team at school can be
    utilized
  • School Psych must be on-site 1 day/week
  • Developing master schedule for Tier 1, 2, and 3
    intervention times is useful
  • Integrate efforts with evaluation referral team
    efforts (consider major reduction in meeting time
    and shift to intervention efforts!)

138
Materials needed
  • Computer and software to organize data
  • Student data imported. Clerical person to enter
    data on-site for tier 1 screen only.
  • Color printer to print graphs extra color
    cartridges
  • Probe materials, digital count-down timers
  • Intervention protocols, intervention materials
    (e.g., flashcard sets, reading materials)
  • Access to copier and some assistance with copying
  • Reinforcers for treasure chest (no more than 500
    per school)

139
Guidelines for Implementers
  • Use single trial scores for screening
  • Following screening, grade-wide graphs to
    principal
  • Return data to teachers within 48 hours with
    personal interpretation at grade-level team
    meeting
  • Include principal in critical meetings
  • Involve teachers at all stages

140
Guidelines for Implementers
  • Learn about curriculum and instruction.
  • Integrate RTI with ongoing school and system
    reform efforts
  • Thoughtfully merge to subtract duplicate
    activities and to enhance more comprehensive
    supplemental and core instructional support
    activities that may be in place
  • Use RTI data to evaluate the value of ALL
    instructional programs or resource allocation
    decisions. Quantify bang for the buck using
    student performance data.

141
Lessons Learned from Vail USD
  • Infrastructure for education being a
    results-based enterprise
  • Accountability
  • Principal is the Instructional Leader of a school
  • Principal as change agent
  • School psychologist as change agent
  • Replace resources/substitute dont add
  • Minimize meetings
  • Track outcomes that matter

142
Upset parent
Tantruming child
Bus 11 is late
Police on site for child abuse report
Principal
Visit from health department, bathroom out of
paper towels again
Teacher out sick for rest of year
Meeting at district office/items due
143
Principal
FILTER-- How much time allocated to instruction?
Children actively engaged? Standards introduced?
Effective instruction occurring?
Upset parent
DATA on Learning
Check on health dept
Goal Setting
Check on police interview
Teacher Evaluation
Etc.
Allocation of Instructional Resources
144
Great Instructional Leaders
  • Have a filter
  • Allocate time and resources according to their
    filter
  • Use an AIMLINE
  • Have a framework for making data-driven decisions
    (know how to access the data they need to reach
    timely decisions)
  • Hold teachers, staff, students accountable
  • Research findings on effective schooling

145
Great School Psychologists
  • Hand the principal the data the principal did not
    know to ask for but cant live without
  • Follow the aimline and attend to implementation
    integrity
  • Understand the variables of effective instruction
    and engage in contextualized assessment that is
    technically valid for the purposes needed AND has
    treatment utility
  • Minimizes meeting time and avoids the science of
    strange behavior

146
Great Districts
  • Minimize time away from school, but use time
    together to review school improvement
    implementation efforts and ongoing results
  • Have the will to proactively chart the course of
    a district
  • Provide adequate resources and space for
    principals to be effective instructional leaders
    and hold them accountable for results
  • Respect the role of parents and actively engage
    them
  • Have a framework for evaluating results (know how
    to access data for decision making)
  • Evaluate quality of all programs locally and make
    decisions about continued use based on DATA.

147
Great Teachers
  • Use data to identify where more/different/less
    instruction is needed
  • Have as a goal to accelerate all learning of all
    children
  • Proactively address barriers to learning
  • Take responsibility for learning that occurs in
    the classroom
  • Are confident and ready to collaborate in the
    classroom
  • Appreciate childhood and children (a little
    humor, lots of patience, enthusiasm)

148
For More Information
  • amandavande_at_gmail.com
  • www.isteep.com
  • Thank you to the US Dept of Education for
    providing all film clips shown in this
    presentation
Write a Comment
User Comments (0)
About PowerShow.com