Application and Evaluation of Positive Behavior Support in Two Urban Elementary Schools Sheila M' Cl

1
Application and Evaluation of Positive Behavior
Support in Two Urban Elementary SchoolsSheila
M. Clonan, Gretchen E. Lopez, Gretchen K.
Rymarchyk, Sigrid M. Davison

• Syracuse University Violence Prevention Project
• School of Education, Syracuse, NY 13244
• [phone]
• smclonan_at_syr.edu
• Annual Meeting of American Educational Research
  Association
• April 12-16, 2004
• San Diego, CA

2
Previous Research

• While there is considerable knowledge about
  effective school interventions, serious
  challenges remain at level of intervention and
  practice
• e.g., Shinn, Walker Stoner, 2002 Tolan, 2002
• Early prevention programs are less costly and
  more effective
• e.g., Arizona Prevention Resource Center, 2003
  Fox, Dunlap, Cushing, 2002
• Prevention programs are needed at elementary
  school level given increase in antisocial
  behavior from elementary to middle school, and
  from middle to high school
• e.g., Fox, Dunlap Cushing, 2002
• Need programs to create change in school culture
  and emphasize positive student behaviors

3
Syracuse UniversityViolence Prevention Project

• Design, implement, and evaluate violence
  prevention efforts within the Syracuse City
  School District
• Collaborate with, and support, 3 elementary
  schools
• 2 implementing a school-wide positive behavior
  program (PBIS) as part of a state initiative
• 1 strengthening school-wide program to develop
  and support, research, and evaluation, ongoing
  consultation and coaching
• Provide assistance with training and intervention
  support, research and evaluation, ongoing
  consultation and coaching
• Emphasis on facilitating evidence-based decision
  making to identify what is most effective
• Compare implementation and outcomes across
  schools to identify elements that help or hinder
  effectiveness

4
City and community context Problem, challenges,
and strengths

• Trend of increasing youth violence
• 155 increase in youth gun violence from 1999 to
  2000
• Record-breaking number of homicides in 2002
• Signs trend is continuing
• Area of concentrated poverty
• Schools located in area with poverty rates 4x
  higher than city average
• Especially high rates of poverty for African
  American and Latino children
• Numerous community-based organizations,
  resources, efforts
• Face national and local level funding cuts, and
  need for greater coordination of services
• Need for early prevention efforts, in addition to
  focusing on youth most at risk

5
School Profiles Student DemographicsNY State
School Report Card 2003

• Bryant
• Grades K-5
• 54 staff
• 513 students
• 89 free or reduced lunch
• Over 81 receive public assistance
• 95 African American
• 4 White
• 2 Hispanic

• Addison
• Grades K-6
• 65 staff
• 665 students
• 85 free or reduced lunch
• Over 81 receive public assistance
• 85 African American
• 13 White
• 2 Hispanic

6
NYS Achievement Test English Students Pass
7
NYS Achievement TestMath Students Pass
8
Positive Behavioral Interventions and Supports
PBIS

• School-wide model, multi-level approach, with
  emphasis on prevention
• Lewis, Sugai, Colvin (1998)
• Students receive positive attention for positive
  behavior
• Team of school staff sets behavioral
  expectations, recognition and rewards, responsive
  to schools specific needs
• Expectations are clearly defined for students,
  taught at start of school year, and reinforced
  throughout the year
• Team collects and analyzes student behavior data
  and modifies environment to respond

9
School-Wide Support Systems

• 3 levels of intervention
• Intensive, individual
• Assessment-based
• Intense procedures
• 1-5
• Targeted, group
• Some students, at risk
• High efficiency, rapid response
• 5-10
• Universal
• All settings, all students
• Preventive, proactive
• 80-90

10
PBIS Implementation

• Year 1
• Universal level
• Year 2
• Universal level
• Directed intervention and trainings Bus and
  classroom
• Targeted level
• Intensive level (in planning stages)

11
PBIS Behavioral Expectations

• Addison
• Behavioral expectations
• Be respectful, be responsible, be safe
• Settings
• Hall, cafeteria, bathroom, arrival and dismissal,
  auditorium, bus
• Bryant
• Behavioral expectations
• Be respectful, be here and ready, be safe, be
  caring
• Settings
• Classroom, hall, cafeteria, bathroom, arrival and
  dismissal, playground

12
Behavioral Expectations for Classroom Bryant
13
Behavioral Expectations for Hall Addison
14
Examples ofPositive Behavior Rewards

• Addison
• Pawprints
• 5stickers
• 10pencils
• 25ice cream
• 50special area time
• 75lunch with administrator
• 100t-shirt

• Bryant
• Gotcha cards
• 3pencil
• 5gel pen
• 10ice cream
• 25BK certificate
• 50picture in school Gotcha Hall of Fame

15
Total Referrals by MonthYear 1 for Bryant and
Grade 5
16
Bus Driver Breakfast

• Data from Y1 indicated directed intervention
• Trainings for this target group were limited
• Sources available did not match our needs, or
  limited applicability
• Six meetings with bus drivers planned in Y2
• Intro to PBIS
• Integrate bus drivers in school culture
• Encourage bus driver participation
• Strengthen bridge between administration and bus
  drivers
• Share ODR data
• Target specific student behaviors

17
Methods Measures

• Student behavior
• Suspensions and attendance
• Office discipline referrals (ODR)
• Ratings from teachers, staff, and teams
• Surveys of teachers and staff
• Surveys of PBIS teams
• Focus groups with teachers and staff
• Observations and interviews with teams, staff,
  and administration

18
Methods Measures

• Surveys of teachers and staff
• School safety Risk and protective factors
• Oregon School Safety Survey
• Sprague, et al. (1995)
• Problem and pro-social student behaviors
• SUVPP
• Acceptability and integrity of implementation
• Intervention Rating Scale
• Adapted from Martens, et al. (1985)
• McDougal, Clonan, Martens (2000)
• Ratings of PBIS Implementation
• SUVPP

19
Methods Measures

• Surveys of PBIS teams
• Status of implementation and priorities for
  improvement
• Effective Behavior Support Survey (EBS)
• Sugai, et al. (2000)
• Focus groups with teachers and staff
• Strengths, limitations, and suggestions for
  further implementation

20
Figure 1School-Wide Support Systems

• 3 Levels of Intervention
• Elementary School in Good Shape
• 1-5 Intensive
• 5-10 Targeted
• 80-90 Universal
• Addison - Year 1
• 16 Intensive
• 16 Targeted
• 68 Universal

21
Figure 2Average Referrals Per Day Year 1 Year
2
22
Figure 3Referrals by Type of Behavior Year 1
23
Figure 4Referrals by Type of Behavior September
to January
24
Figure 5Referrals by LocationYear 1
25
Figure 6Average Referrals Per Day for Classroom
September to February
26
Figure 7Average Referrals Per Day for Bus
September to February
27
Table 1School Safety RiskResponse scale 1 not
at all to 4 extensively

• Addison M
• Year 1 Fall (n27)
• Poverty 3.85
• Fights 3.58
• Bullying 3.56
• Year 2 Fall (n38)
• Poverty 3.68
• Fights 3.47
• Bullying 3.32

28
Table 2School Safety ProtectiveResponse scale
1 not at all to 4 extensively

• Addison M
• Year 1 Fall
• Acceptance of diversity 3.23
• Professional development 3.16
• Supervision of students across settings 3.04
• Year 2 Fall
• High expectations for student learning 3.59
• Professional development 3.38
• Effective student-teacher relationships 3.32

29
Table 3Ratings of problem and pro-social
behaviorsResponse Scale 1 strongly disagree to
6 strongly agree
30
Table 4Intervention Rating ScaleResponse Scale
1 strongly disagree to 6 strongly agree
31
Figure 8Status Priority by Area Addison
Status response scale 1 not in place to 3 in
place Priority response scale 1 low priority to
3 high priority
32
Table 5High Priority for Improvement PBIS Team
Member Ratings of Implementation (n7)
Response scale 1 low priority, 2 medium
priority, 3 high priority
33
Table 6Focus Groups with Teachers and Staff
Year 1 Addison

• Benefits
• Gives the school a common language
• Challenges
• Lack of consistent implementation across
  teachers, and across locations
• Does not address angry and violent behavior
  problems or students with severe or chronic
  problems
• Ideas to Improve
• School-wide boosters
• More teacher training in classroom management
• Need parent involvement - e.g., PBIS newsletter
  to parents

34
Figure 9School-Wide Support Systems

• 3 Levels of Intervention
• Elementary School in Good Shape
• 1-5 Intensive
• 5-10 Targeted
• 80-90 Universal
• Bryant - Year 1
• 8 Intensive
• 14 Targeted
• 78 Universal

35
Figure 10Average Referrals Per Day Year 1
Year 2
36
Figure 11Referrals by Type of Behavior September
to January
37
Table 7School Safety RiskResponse scale 1 not
at all to 4 extensively

• Bryant M
• Year 1 Fall (n43)
• Poverty 3.79
• Fights 3.53
• Bullying 3.45
• Year 2 Fall (n37)
• Poverty 3.81
• Gang activity 3.51
• Fights 3.50

38
Table 8School Safety ProtectiveResponse scale
1 not at all to 4 extensively

• Bryant M
• Year 1 Fall
• Acceptance of diversity 3.23
• Professional development 3.21
• High expectations for student learning 3.19
• Year 2 Fall
• High expectations for student learning 3.35
• Supervision of students across settings 3.29
• Student support services 3.25

39
Table 9Ratings of problem and prosocial
behaviorsResponse Scale 1 strongly disagree to
6 strongly agree
40
Table 10Intervention Rating ScaleResponse
Scale 1 strongly disagree to 6 strongly agree
41
Figure 12Status Priority by Area Bryant
Status response scale 1 not in place to 3 in
place Priority response scale 1 low priority to
3 high priority
42
Table 11High Priority for Improvement PBIS Team
Member Ratings of Implementation (n6)
Response scale 1 low priority, 2 medium
priority, 3 high priority
43
Figure 13Average Referrals Per DayPer Student
44
Table 12Ratings of PBIS implementationResponse
Scale 1 highly dissatisfied to 6 highly satisfied
45
Summary and Implications

• Studied and evaluated first and second year (in
  progress) of PBIS implementation at two urban
  elementary schools
• Evidence of effectiveness at first school
• Decrease in referrals most problematic behaviors
  and locations
• Teachers and staff offer favorable ratings of
  PBIS implementation - including acceptability and
  integrity
• Issues to address include (1) targeted and
  intensive levels of intervention, (2) greater
  consistency in implementation, (3) need for
  parent involvement
• Less evidence of effectiveness at second school
• Teachers and staff offer somewhat favorable
  ratings of PBIS implementation
• Ratings of PBIS implementation are less
  favorable, however, than ratings from teachers
  and staff at first school (Year 1, Spring)

46
Summary and Implications

• Issue of fidelity of implementation
• How best to measure this and its impact or
  consequences for outcomes
• How best to balance schools needs and challenges
  with progress toward effective change
• Importance of research and evaluation
• Understanding longitudinal effects
• Need 3-5 years to see organizational change
• Multiple measures of outcomes