Intervention Implementation Integrity Within Conjoint Behavioral Consultation: Strategies to Maximize Outcomes

1
Intervention Implementation Integrity Within
Conjoint Behavioral Consultation Strategies to
Maximize Outcomes

• S. Andrew Garbacz, MA
• Susan M. Sheridan, PhD
• Michelle S. Swanger-Gagne, MA
• Amanda L. Witte, MA
• University of Nebraska-Lincoln
• The Nebraska Center for Research on Children,
  Youth, Families, and Schools (CYFS
  http//cyfs.unl.edu)
• The project is funded by the Institute of
  Educational Sciences
• Grant Award Number R305F050284

CBC in the Early Grades
2
School-Based Consultation

• School-based behavioral consultation is an
  indirect form of service delivery wherein a
  specialist with expertise in social-behavioral
  and academic interventions works collaboratively
  with a treatment agent (i.e., consultee, such as
  a teacher or parent).
• School-based consultation may focus on
  behavioral, socio-emotional, or academic
  concerns, which interfere with learning.

3
Conjoint Behavioral Consultation

• Conjoint behavioral consultation (CBC Sheridan,
  Kratochwill, Bergan, 1996 Sheridan
  Kratochwill, 2008) is an extension of traditional
  behavioral consultation services.
• CBC is, by definition, an indirect model of
  service delivery intended to enhance the skills
  of natural treatment agents (i.e., teachers,
  parents) who assume responsibility for
  implementing intervention plans.
• Goals of CBC focus on addressing the specific
  needs of the child, while working collaboratively
  with the childs teacher and parent to
  strengthen family-school partnerships.
• - Including enhancing the skills of natural
  treatment agents (i.e., consultees).

4
Implementation Integrity

• Intervention implementation integrity refers to
  the extent to which an intervention plan is
  implemented as intended.
• An important consideration in CBC is the ability
  of individuals to deliver treatment plans
  accurately and efficiently.
• Thus, the issue of implementation integrity is
  central to the utility of CBC in addressing
  students behavioral needs.

5
Implementation Integrity

• Little systematic attention is afforded to the
  specific strategies that consultants can use to
  ensure accurate and reliable implementation of
  treatment plan elements in natural settings
  (Sanetti Kratochwill, 2008).
• Strategies to support families in implementing
  interventions within a problem-solving framework
  have not received adequate attention.

6
Socioeconomic Conditions

• A significant correlate with treatment dropout
  and poor outcomes following parent training
  interventions is socioeconomic hardship
  (Henggeler et al., 1992 Kazdin, 1990 McMahon et
  al., 1981).
• Families living in low socioeconomic conditions
  face numerous stressors (e.g., poverty, lack of
  social support) that may interfere with their
  ability to implement parenting interventions,
  thereby compromising treatment effects.

7
Purpose

• Describe the degree to which families implement
  interventions as designed in a consultation study
  to promote positive behavioral outcomes for
  children experiencing behavioral difficulties.
• Describe and illustrate multiple methods of
  assessing integrity.
• Discuss strategies used to maximize intervention
  implementation integrity of treatment plans by
  all families to increase positive outcomes.
• Present methods used by consultants to support
  families with diverse challenges.

8
Methods - Participants

• Child Participants
• 62 children
• 75 Male
• 67 White, non-Hispanic
• Average age 6.72 (range 5 9)
• All students were referred to the study by their
  teachers who indicated a need for further
  intervention due to behavioral concerns.

9
Methods - Participants

• Parent Participants
• 62 Parents
• 83 White, non-Hispanic
• Average age 35
• 88 Female
• 78 received some college or post-secondary
  education
• 26 parents met the criteria to be included in a
  demographically diverse sub-sample
• Data used in this investigation represent part of
  a larger, four-year study evaluating the efficacy
  of CBC as an intervention that addresses concerns
  of kindergarten through third grade students
  whose disruptive behaviors place them at risk for
  academic failure.

10
Methods - Setting

• Cases were conducted in 17 kindergarten, first,
  second, and third grade classrooms in a
  moderately-sized Midwestern city and surrounding
  area, including public and parochial schools.
• Interventions were implemented across home and
  school settings for students who were the focus
  of consultation.

11
Methods - Selection

• Students were selected via teacher ratings
• Systematic Screening for Behavior Disorders
  (SSBD) multiple-gate screening procedure (Walker
  et al., 1990).
• Screening instrument with ratings for severity of
  externalizing behaviors frequency of
  externalizing behaviors and degree of need for
  additional intervention.

12
Methods Data Procedures

• Cases were separated into two groups
• General mainstream group (N36)
• Demographically diverse group (N26)
• Relative to mainstream group, demographically
  diverse was defined as low-income status,
  racially diverse, linguistically diverse, living
  in a single parent home, or less than high school
  diploma.

13
Consultation Procedures

• Consultants (i.e., trained graduate students) met
  with the teacher and two to three parents for
  approximately four to five conjoint consultation
  sessions over approximately eight weeks via three
  phases.
• 1) Needs Identification and Needs Analysis
• 2) Intervention Development and Implementation
• 3) Intervention Evaluation
• Behavioral interventions consisted of 3
  components.
• 1) Communication component
• 2) Motivation component
• 3) Functional component

14
Implementation Support

• Strategies to promote high integrity in the home
  setting were developed as part of the larger
  study
• Sources
• 1) Literature review that informed consultant
  training.
• 2) Narrative information reported by trained
  consultants
• See Handout 1

15
Implementation Support

• General Strategies
• Follow the partnership model during intervention
  development phase (Sheridan Kratochwill, 2008).
• Provide a rationale for collecting integrity
  data.
• Script and package intervention implementation
  integrity forms (Watson, 2004).
• Train and educate the family on intervention
  delivery with various techniques.
• See handout 1

16
Implementation Support

• Additional strategies for families requiring more
  support
• 1) Consider culturally-sensitive procedures
  (Sheridan Kratochwill, 2008).
• 2) Focus on strategies that build trust between
  families and educators.
• 3) Enhance communication between consultant and
  families.
• 4) Implement family-centered approach throughout
  consultant process.
• 5) Gather information about the home setting and
  family system.
• 6) Facilitate regular contacts and provide
  collective support.
• 7) Adjust integrity data collection as to improve
  ease of collection.
• See handout 2

17
Case Study

• 6 year old female in 1st grade was the focus of
  consultation.
• Biological parents and 4 siblings moved to the
  United States from Sudan 7 years prior as
  refugees.
• The family currently meets the 2008 HHS criteria
  for a family living in poverty.

18
Case Study - Preconsultation

• Conducted an initial meeting to introduce
  consultation.
• Validated parent concerns (e.g., acculturating to
  the United States, feedback from classroom
  teachers).
• Met with a cultural liaison about roles,
  structure, and expectations.
• Conducted a home visit to discuss roles and
  expectations.

19
Case Study Needs Identification and Analysis

• Strengths
• Strong verbal skills
• Independent
• Concerns
• Reading skills
• Ability to stay on-task disruptive behaviors
• Functional assessment revealed escape motivated
  behavior, and grade level academic skills.

20
Case Study Needs Identificationand Analysis

• Intervention development
• School chunking reading work, appropriate
  escape, self-monitoring.
• Home structured homework routine that included
  entire family, home-school note, activity
  checklist with visual cues.
• Family checklist to increase integrity.
• Additional support (e.g., meeting reminders,
  frequent communication).

21
Measurement of Treatment Integrity

• Parent Self-Report Form
• Completed daily by parents and used to assess
  intervention implementation integrity at home.
• Listed all the steps of the behavioral
  intervention plan clearly and objectively.
• Example includes intervention plan steps that the
  parent in this case completed
• See handout 3

22
(No Transcript)
23
Measurement of Treatment Integrity

• Permanent Product Report Form
• Completed by parents daily for the duration of
  intervention (i.e., at least 4 weeks of
  intervention).
• Charts were collected from parents on which
  evidence (e.g., stickers, notes, marks, checks)
  demonstrating that s/he implemented steps of the
  intervention.
• Trained research assistants and consultants
  reviewed permanent products and completed the
  permanent product report forms to reflect
  parents delivery of plan components as reported
  on permanent products.
• See handouts 4 and 5

24
Activity Checklist
25
(No Transcript)
26
Case Study Strategies to Maximize Integrity

• Integrity information was scripted and
  standardized
• Intervention steps were simple and consistent
• Opportunity to have forms translated to Arabic
  use of visual cues.
• Home visits for modeling, observation,
  performance feedback, and discussion.
• The family reported adhering to 100 of
  intervention steps over 4 weeks of implementation.

27
Case Study Intervention Evaluation

• School
• Increase from an average of 45 accuracy prior to
  the intervention to 80 accuracy after
  implementation.
• Home
• Decrease in the number of problem behaviors
  exhibited from an average of 4 to an average of
  1.
• Parents indicated improved behavior management
  strategies, communication with the school, and
  praising children for attaining goals.

28
Assessment of Implementation
Table 3 Summary Statistics of Intervention
Implementation Integrity across Sample
Parent (self)a report Permanent Products
Mainstream Group
Percent .77 .79
Standard Deviation .26 .24
Sample Size 26 28
Diverse Group
Percent .81 .91
Standard Deviation .17 .18
Sample Size 19 21
a Participants completed self-report report
measure, permanent product measure, or both
measures
29
Discussion

• High levels of intervention implementation
  integrity were noted in the home setting across
  multiple sources (i.e., families via
  parent-report and coders of permanent product
  data)
• Families seem to be responsive to tactics used by
  consultants to maximize treatment integrity.
• Families also appear to adhere to the steps of
  behavioral interventions regardless of the
  integrity measure used, with families reporting
  slightly higher integrity when recording
  integrity on a permanent product.

30
Discussion

• Permanent products are a feasible and useful way
  of measuring treatment integrity because families
  naturally use the products as they implement
  interventions (Sanetti Kratochwill, 2008).
• High integrity levels reported on permanent
  products may suggest that parents are not
  over-reporting on self-report measures.
• Families at-risk reported implementing
  intervention with higher integrity than the
  mainstream group of families.

31
Implications

• To maximize outcomes, educators may find it
  helpful to use a collaborative consultation model
  to design and implement behavioral interventions
  with families.
• In CBC, training, supporting, and partnering with
  teachers and families provides a consistent and
  continuous approach to service delivery.

32
Future Research Directions

• Examine integrity and the variables that may
  influence levels of treatment integrity
  experimentally.
• Specifically, future research could attempt to
  measure the impact of disadvantaged social
  settings on the degree to which families follow a
  behavioral intervention protocol.
• Explore resources that are needed and specific
  procedures that are critical to intervention
  implementation. The cost-effectiveness of such
  practices needs to be assessed in future studies.
  

33
Future Research Directions

• Measure and evaluate treatment integrity in other
  ways.
• For example, the dosage, quality of, and
  responsiveness to implementation may be important
  in understanding the quality of services students
  are receiving (Dane Schneider, 1998).
• Standardized measures of consultant strategies
  for maximizing integrity should be developed and
  used in future research.

34
Future Research Directions

• Psychometric properties of future measures of
  integrity and strategies used by consultants to
  improve integrity should be examined (e.g.,
  validity, reliability).
• Similar issues with regard to measurement and
  evaluation should be explored at school.
• The link between home and school treatment
  integrity and its effects on student performance
  requires examination.

35
Contact Information

• Susan M. Sheridan ssheridan2_at_unl.edu
• Andy Garbacz agarbacz2_at_unl.edu
• Amanda Witte awitte2_at_unl.edu
• http//cyfs.unl.edu

36
References

• Dane, A. V. Schneider, B. H. (1998). Program
  integrity in primary and early secondary
  prevention Are implementation effects out of
  control? Clinical Psychology Review, 18, 23-45.
• Henggeler, S. W., Melton, G. B., Smith, L. A.
  (1992). Family preservation using Multisystemic
  Therapy An effective alternative to
  incarcerating serious juvenile offenders. Journal
  of Consulting and Clinical Psychology, 60,
  953-961.
• Kazdin, A. E. (1990). Premature termination from
  treatment among children referred for antisocial
  behavior. Journal of Child Psychology and
  Psychiatry, 31, 415-425.
• McMahon, R. J., Forehand, R., Griest, D. L.,
  Wells, K. C. (1981). Who drops out of treatment
  during parent behavioral training? Behavioral
  Counseling Quarterly, 1, 79-85.
• Sanetti, L. M. H., Kratochwill, T. R. (2008).
  Treatment integrity in behavioral consultation
  Measurement, promotion, and outcomes.
  International Journal of Behavioral Consultation
  and Therapy, 4, 95-113.

37
References

• Sheridan, S. M., Kratochwill, T. R. (2008).
  Conjoint behavioral consultation Promoting
  family-school connections and interventions (2nd
  ed.). New York, NY Springer Publishing.
• Sheridan, S. M., Kratochwill, T. R., Bergan, J.
  R. (1996). Conjoint behavioral consultation A
  procedural manual. New York Plenum.
• Walker, H.M. Severson, H. H., Todis, B.J.,
  Pedego, A.E. (1990). Systematic screening for
  behavior disorders (SSBD) Further validation,
  replication, and normative data. RASE Remedial
  Special Education, 11, 32-46.
• Watson, T. S. (2004). Treatment integrity. In T.
  S. Watson C. H. Skinner (Eds.), Encyclopedia of
  school psychology (pp. 356-358). New York, NY
  Springer Publishing.