Title: Data-based Decision-making: Evaluating the Impact of School-wide Positive Behavior Support
1Data-based Decision-makingEvaluating the Impact
of School-wide Positive Behavior Support
- George Sugai, Rob Horner, Anne Todd, and Teri
Lewis-Palmer - University of Oregon
- OSEP Funded Technical Assistance Center
- www.pbis.org
2Purpose
- Examine the extent to which the logic of
School-wide Positive Behavior Support (PBS) fits
your real experience in schools - Define the outcomes for School-wide PBS
- Is School-wide PBS related to reduction in
problem behavior? - Is School-wide PBS related to improved school
safety? - Is School-wide PBS related to improved academic
performance? - Define tools for measuring School-wide PBS
outcomes - Examine a problem-solving approach for using
office discipline referral (ODR) data for
decision-making - Provide strategies for using data for
decision-making and action planning
3To Improve Schools for Children
- Use evidence-based practices
- Always look for data of effectiveness
- Never stop doing what is working
- Implement the smallest change that will result in
the largest improvement - Measure Compare Improvement
4Designing School-Wide Systems for Student Success
1-5
1-5
5-10
5-10
80-90
80-90
5SW Positive Behavior Support
Social Competence, Academic Achievement, and
Safety
OUTCOMES
Supporting Decision- Making
DATA
Supporting Staff Behavior
SYSTEMS
PRACTICES
Supporting Student Behavior
6Improving Decision-Making
Solution
Problem
From
Problem-solving Information
Problem
Solution
To
7Problem-solving Steps
- Define the problem(s)
- Analyze the data
- Define the outcomes and data sources for
measuring the outcomes - Consider 2-3 options that might work
- Evaluate each option
- Is it safe?
- Is it doable?
- Will it work?
- Choose an option to try
- Determine the timeframe to evaluate effectiveness
- Evaluate effectiveness by using the data
- Is it worth continuing?
- Try a different option?
- Re-define the problem?
8Key Features of Effective Data Systems
- Data are accurate
- Data are very easy to collect
- Data are used for decision-making
- Data are available when decisions need to be made
- Data collectors must see the information used for
decision-making
9Guiding Considerations
- Use accessible data
- Handle data as few times as possible
- Build data collection into daily routines
- Establish and use data collection as a
conditioned positive reinforcer - Share data summaries with those who collect it
10Types of Questions
- Initial Assessment Questions
- What type or which program do we need?
- Where should we focus our efforts?
- Ongoing Evaluation Questions
- Is the program working?
- If no,
- Can it be changed?
- Should we end the program?
- If yes,
- Do we need this program anymore?
- What do we need to do to sustain success?
11What Data Should be Collected?
- Always start with the questions you want to
answer - Make data that will answer your question
- Easy, available, reliable
- Balance between reliability and accessibility
- Systems approach
- Consider logistics
- Who? When? Where? How?
- Two levels
- What is readily accessible?
- What requires extra resources?
12When Should Data Decisions Be Made?
- Natural cycles, meeting times
- Weekly, monthly, quarterly, annually
- Level of system addressed
- Individual daily, weekly
- School-wide monthly, quarterly
- District/ Region
- State-level
13Basic Evaluation Questionsby School or Program
- What does it look like now?
- How would we know if are successful?
- Are we satisfied with how it looks?
- YES
- Celebrate
- NO
- What do we want it to look like?
- What do we need to do to make it look like
that? - What can we do to keep it like that?
14Basic School-wide PBS Evaluation Questionsby
School/ District/ Region
- Are our efforts making a difference?
- How many schools have adopted School-wide PBS?
- Are schools adopting School-wide PBS to
criterion? - Are schools who are implementing School-wide PBS
perceived as safe? - Are teachers delivering instructional lessons
with fidelity as planned? - Is School-wide PBS improving student outcomes?
15Is School-wide PBS Having a Positive Influence on
School Culture? Using Office Discipline Referral
Data
16Office Discipline Referrals and The BIG 5!
- Examine office discipline referral rates and
patterns - Major Problem events
- Minor Problem events
- Ask the BIG 5 questions
- How often are problem behavior events occurring?
- Where are they happening?
- What types of problem behaviors?
- When are the problems occurring?
- Who is contributing?
17what
where
The BIG 5
when
How often
who
18Office Discipline Referral Caution
- Data reflects 3 factors
- Students
- Staff members
- Office personnel
- Data reflects overt rule violators
- Data is useful when implementation is consistent
- Do staff and administration agree on
office-managed problem behavior verses
classroom-managed behavior?
19(No Transcript)
20General Procedure for Dealing with Problem
Behaviors
Observe problem behavior
Find a place to talk with student(s)
Is behavior major?
NO
YES
Ensure safety
Problem solve
Write referral and Escort student to office
Problem solve
Determine consequence
Determine consequence
Follow procedure documented
Follow documented procedure
Does student have 3?
NO
YES
Follow through with consequences
File necessary documentation
Send referral to office
File necessary documentation
Follow up with student within a week
21(No Transcript)
22SWIS Compatibility Checklist Procedure for
Documenting Office Discipline Referrals School
___________________________
Date ____________________
Next review date _______________ Redesign
your form until answers to all questions are
Yes. Readiness requirements 4 and 5 are
complete when you have all Yes responses.
23Tables versus Graphs
24 25Number of ODR per Day and Month
26Total verses Rate
27Total Number of ODRs per Month
28Number of ODRs per Day and Month
29Priorities and Rationale
30SWISTM summary 05-06 (Majors Only)1668 schools,
838,184 students
31Interpreting Office Referral DataIs there a
problem?
- Absolute level (depending on size of school)
- Middle Schools (gt5 per day)
- Elementary Schools (gt1.5-2 per day)
- Compare levels to last year
- Improvement?
- Trends
- Peaks before breaks?
- Gradually increasing trend across year?
32- What is summary for
- High school of 850 students? 2. High School of
1825 students? - 3. Elem. school of 625 students? 4. Middle
School of 625 students?
33- What is summary for
- High school of 850 students? 2. High School of
1825 students? - 3. Elem. school of 625 students? 4. Middle School
of 625 students?
34Trevor Test Middle School
- 565 students
- Grades 6,7, and 8
35Lang. Defiance Disrespect
Harrass Skip
1200
Cafeteria Class Commons Hall
36Langley Elementary School
- 478 Students
- Kindergarten - Grade 5
37(No Transcript)
38(No Transcript)
39(No Transcript)
40(No Transcript)
41What Does a Reduction of 850 Office Discipline
Referrals and 25 Suspensions Mean?Kennedy
Middle School
- Savings in Administrative Time
- ODR 15 minutes/ event
- Suspension 45 minutes/event
- 13,875 minutes
- 231 hours
- 29, 8-hour days
- Savings in Student Instructional Time
- ODR 45 minutes/ event
- Suspension 216 minutes/ event
- 43,650 minutes
- 728 hours
- 121, 6-hour school days
42Is Implementation Related to Reduction in Problem
Behavior?767 students
43(No Transcript)
44(No Transcript)
45(No Transcript)
46(No Transcript)
47Are Schools Adopting School-wide PBS to
Criterion?
- Use the
- Team Implementation Checklist (TIC)
- School-wide Evaluation Tool (SET)
- EBS Self-Assessment Survey (School-wide section)
- Other
- Measure and analyze annually
- Well focus on TIC today!
48Team Implementation Checklist (TIC)
- Characterizes the evolution of School-wide PBS
implementation - Achieved, In progress, or Not started
- Assists in
- Initial assessment
- Getting started on action plan
- Measuring progress of School-wide PBS
Implementation - Assesses team-based response
- Quarterly or monthly
49TIC Feature Areas
- Establish Commitment
- Establish and Maintain Team
- Conduct Self-Assessment
- Define Expectations
- Teach Expectations
- Establish Reward System
- Establish Violations System
- Establish Information System
- Build Capacity for Function-based Support
- Ongoing Activities
50Team Implementation Checklist
51Team Implementation Checklist continued
52Team Implementation Checklist continued
53Scoring the TIC
- Implementation Points
- Achieved 2
- In progress 1
- Not Started 0
- Percentage of Items Implemented
- Total
- Number of items scored as Achieved divided by
17 (items) - Subscale scores
- Number of items in each subscale area scored as
Achieved divided by the number of items in that
subscale area - Percentage of Points Implemented
- Total
- Total number of points divided by 34
- Subscale scores
- Total number of points in each subscale divided
by total number of items multiplied by 2
54Team Implementation Checklist (TIC)
55Team Implementation Checklist (TIC)
56Total Average TIC Scores
Schools
571. What is working well? 2. What are next steps?
581. What is going well? 2. What are next steps?
591. What is going well? 2. What are next steps?
60A Case Study Example
61Patterson and Family Schools
- K-5, 250 students
- Traditional and alternative schools share a
campus - Organized for PBS in 1996
- First year implementation in 1997
- One administrator one Site Council/ Advisory
Board two budgets - Sustained efforts through 2004
- New school built, two campus merged
- PBS continued
- Staff and families would have it no other way!
62(No Transcript)
63Pre score not available
64(No Transcript)
65(No Transcript)
66(No Transcript)
67(No Transcript)
68Eugene School District
- Assessment of academic gains across five years
(1997-98 to 2001-02) - Proportion of students meeting state standards in
reading and math - Change in percentage of students meeting state
standards - Pre 1997-98
- Post 2001-02 (four years of implementation)
69(No Transcript)
70(No Transcript)
71Main Messages
- Invest in prevention
- Create an effective environment
- Leadership, teams hosts for effective practices
- Use different systems for different problems
- Individual student level alone will be
insufficient - Collaboration with Mental Health Professionals
- Build a culture of competence
- Define, teach, monitor, and reward appropriate
behavior - Build sustainable systems
- Resist person-dependent interventions
- Invest in gathering and using information for
decision-making and problem-solving
72Action Planning
- Use your self-assessment information
- Rally School-wide commitment
- Establish a PBS Team
- Focus on prevention (define, teach, monitor, and
reward appropriate behavior) - Ask kids tomorrow if they know the expectations
- Ask kids if they are being acknowledged for
appropriate behavior - Use information system to guide implementation
efforts - Build Action Plan
- When will the team meet?
- What will be reported to faculty?
- What will be reported to families?
73Action Planning
- Which system are you going to work on?
- What are the specific outcomes?
- When will they be completed?
- What short-term activities are needed?
- Who will be responsible?
- Reporting Schedule
- What information will be gathered and by whom?
- When will information be reported?
74(No Transcript)