Tools That Work: Improving Child Welfare Services Through Research, Performance Measurement, and Information Technology November 12, 2003

1
Tools That Work Improving Child Welfare
Services Through Research, Performance
Measurement, and Information TechnologyNovember
12, 2003

• Best Practice Outcomes in Behavior Support and
  Intervention
• A Multi-site Evaluation
• Lloyd Bullard
• Katie Johnson

2
Background

• Physical Risks
• Hartford Courant series documents 142 deaths
  related to the use of restraint and seclusion
• Harvard Center for Risk Analysis estimates 50-150
  deaths per year are related to the use of
  restraint and seclusion

3
Background

• Psychological Risks
• Consumers describe restraint and seclusion as
  dehumanizing and humiliating (Binder and McCoy,
  1983)
• Children and adolescents who have been restrained
  in psychiatric hospitals report nightmares,
  intrusive thoughts, avoidance responses, and
  marked startle responses (General Accounting
  Office, 1999).
• Restraint and seclusion have been identified as
  having the negative neuro-biological effects
  associated with trauma and re-traumatization

4
Background

• Charles Curie and the Pennsylvania Initiative
• Childrens Health Act of 2000
• SAMHSA develops RFP
• Three-year grant beginning October, 2001
• Selects five demonstration sites and one
  coordinating center

5
The Coordinating Center Partnership CWLA and
FFCMH

• Child Welfare League of America
• Establish an Advisory Committee
• Provide Technical Assistance
• Conduct Evaluation
• Disseminate Findings
• Develop Tools and Resources

6
The Coordinating Center Partnership CWLA and
FFCMH

• Federation of Families for Childrens Mental
  Health
• Provide Technical assistance to sites
• Participate on the National Advisory Committee
• Conduct key informant groups and provide
  comprehensive report of focus group activities
• Establish Family Advocate National Advisory
  Committee

7
Demonstration Sites

• The Methodist Home for Children and Youth Macon,
  GA
• Girls and Boys Town National Resource and
  Training Center Boys Town, NE
• A.B. and Jessie Polinsky Childrens Center San
  Diego, CA
• Lakeside Treatment and Learning Center
  Kalamazoo, MI
• University of Alabama Brewer Porch Children's
  Center Tuscaloosa, AL
• Connecticut Collaboration for Training Excellence
  
• Klingberg Family Centers New Britain, CT
• Devereux Glenholme School Washington, CT
• Riverview Hospital Middletown, CT

8
Project Goals

• Year One Identify best practices for reducing
  restraint and seclusion use
• Year Two Reinforce staff training with
  organizational support
• Year Three Disseminate findings

9
Evaluation Methodology

• Individual Incident Data
• Individual variables (e.g. race, gender, time of
  day, incident precipitant, etc.) recorded for
  each incident of emergency physical intervention
• Standardized Measures
• Measures (e.g. Organizational Climate, Family
  Centeredness Assessments, etc.) evaluated at
  specific time periods
• Qualitative Measures
• Reports to SAMHSA, significant events timelines
  process meetings

10
Challenges to Evaluation

• Demonstration Sites
• Data collection worthiness - for the purposes of
  SAMHSA research
• Incident tracking
• Isolating intervention effects
• Over exposure
• Risk management
• Administratively burdensome
• Program rigidity

• Coordinating Center
• Uniformity
• Common definitions
• Common data points
• Common measures
• Complex design
• Multiple sites
• Multiple baselines
• Multiple interventions
• Multiple crises training modules (e.g., TCI, CPI,
  etc.)
• Isolating intervention effects
• Provide context to outcomes

11
Evaluation Tools

• Key Informant Focus Groups
• Organizational Climate Assessment
• 14 dimensions that identify organizational
  factors affecting outcomes
• Assessment of Policy and Procedure
• Based upon Best Practice Guidelines For Behavior
  Management
• Family Centeredness Assessment
• Family-Professional Partnership Scale developed
  by University of Kansas
• Trieschman Carolinas Project Instrument (TCPI)
• Process Assessment and Feedback
• Data Collection

12
Current Evaluation Outcomes

• Key Informant Focus Group Findings
• The seclusion and restraint process needs to be
  explained more thoroughly.
• The training of staff to handle seclusion and
  restraint incidents is critical to avoiding
  injury.
• Family members need to be a part of the service
  planning process.
• Family members need to be notified in a timely
  way of any incidents.
• Youth in facilities need to be able to file
  grievances.

13
Current Evaluation Outcomes

• Organizational Climate Assessment
• Staff from two sites successfully reducing
  restraint and seclusion were rated highly in the
  following categories
• Ability to get things done, meet the needs of
  their clientele, and use funds efficiently.
• Ability to respond to change, flow of
  communication, compensation, competency, morale,
  personnel evaluation process, training,
  organizational values, and mission to serve
  youth.
• Quality of relationship with supervisor and
  perceived standards of services rendered.

14
Current Evaluation Outcomes

• Assessment of Policy and Procedure
• Overall Policy Findings
• The findings revealed that there was little
  difference between the sites staff and the
  external site consultants assessment ratings.
• What Sites are Doing Best
• Consultants and site representatives rated the
  sites the highest on policies governing Legal and
  Ethical concerns and Professional Development.
• What needs Improving
• Representatives and consultants indicated
  complete agreement that the sites were relatively
  weakest on their written policies governing
  Medical Issues.

15
Current Evaluation Outcomes

• Family Centeredness Assessment
• Families indicate that it is very important to
  them that they have good working relationships
  with their service provider.
• Families indicate that they are satisfied with
  their relationship with their childs primary
  service provider.

16
Current Evaluation Outcomes

• Process Assessment and Feedback
• Sites attributed trends in physical intervention
  data to the following
• Temporal Factors
• Policy and Practice Factors
• Staff Factors
• Child Factors
• Training Factors

17
Current Evaluation Outcomes
18
Current Evaluation Outcomes
19
Current Evaluation Outcomes

• Overall Practice Findings
• The frequency of incidents largely coincides with
  seasonal events.
• The use of seclusion and mechanical restraint has
  decreased.
• The number of physical restraint incidents has
  increased sharply over the past six months.
• Overall, injuries to children and staff have
  increased.
• What Sites are Doing Best
• Multiple preemptive behavior management
  techniques are attempted with the youngest
  children.
• There has been no increase in medication
  management as reported by most sites.
• The duration of seclusion and mechanical
  incidents is decreasing.
• What Needs Improving
• Property damage accounts for 20 of all behavior
  management incidents.
• Injuries to staff and child occur more often in
  restraints that require moving the child.
• Ethnic disparities exist in the use of seclusion
  and physical restraint.

20
Program Resources

• CWLA Best Practice in Behavior Support and
  Intervention Assessment Instrument
• Trieschman Carolinas Project Instrument A
  Questionnaire For Family Centered Group Care
  Practice
• Effective Supervisory Practice Behavior Support
  and Intervention for Children and Youth
• National Task Force on Behavior Support and
  Intervention Training Guidelines
• Reducing the Use of Restraint and Seclusion
  Promising Practices and Successful Strategies
• Other Resources

21
Project Resources

• CWLA Best Practice in Behavior Support and
  Intervention Assessment Instrument
• Aims to help agencies improve their behavior
  support and intervention policies, procedures and
  practices through careful self-assessment
• Example
• The agency has a philosophy governing the use of
  restrictive procedures.
• Any behavior management intervention is first
  based on the premise of first, do no harm.
• Nonphysical interventions are always the first
  choice, unless safety issues demand an immediate
  physical response.
• Caregivers are involved in developing and/or
  selecting a behavior intervention model.
• Caregivers support the philosophy governing the
  use of restrictive procedures as indicated by
  their knowledge of that philosophy, by their
  adherence to the philosophy in daily practice,
  and by their completion of all relevant
  documentation.
• A written philosophy governing the use of
  restrictive procedures is present, and it is
  consistently reflected in all areas of agency
  practice.
• A written philosophy governing the use of
  restrictive procedures is present, but only
  partially reflected in agency practice.
• A written philosophy governing the use of
  restrictive procedures is present, but poorly
  reflected in agency practice.
• A written philosophy governing the use of
  restrictive procedures is absent or clearly
  inadequate.
• Not applicable.

22
Program Resources

• Trieschman Carolinas Project Instrument A
  Questionnaire For Family Centered Group Care
  Practice
• A staff survey to assess an agencys commitment
  to family-centered group care
• Example
• To protect the interests of children they must
  remain connected to their parents regardless of
  circumstances.
• Strongly agree Strongly disagree
• (1) (2) (3) (4) (5)

23
Program Resources

• Effective Supervisory Practice Behavior Support
  and Intervention for Children and Youth
• Synthesizes the best practices that supervisors
  can employ in order to reduce the use of
  restraint and seclusion
• Competencies
• Supervisors understand that positive
  organizational climate is necessary to an
  effective behavior management program.
• Supervisors know how to support staff in
  developing the skills and attitudes that are
  essential to successful implementation of an
  effective behavior management program.
• Supervisors understand their role and essential
  skills and attitudes in successful implementation
  of an effective behavior management program.

24
Program Resources

• National Task Force on Behavior Support and
  Intervention Training Guidelines
• Creation of guidelines that will assess behavior
  support and intervention training programs.
• Proposed Chapters
• Organizational Culture and Leadership
• Behavior Support Training Programs
• Risk Factors
• Emergency Physical Interventions
• Training Process

25
Program Resources

• Reducing the Use of Restraint and Seclusion
  Promising Practices and Successful Strategies
• An issue brief that annotates policies and
  practices that successfully reduce the use of
  restraint and seclusion
• Chapters
• Leadership
• Organizational Culture
• Agencies policies, procedures and practices
• Staff Training and Professional Development
• Treatment Milieu
• Continuous Quality Improvement

26
Program Resources

• Website
• Annotated Bibliography
• Standards/Definitions for Restraint and Seclusion
  
• Report Card Vols. 1 to Vol. 5
• CWLA Best Practice Guidelines for Behavior
  Management

27
Recommendations and Next Steps

• Creation of a national resource center that
• Acts as a clearinghouse that creates/disseminates
  evidenced-based practices
• Compiles research and data on restraint and
  seclusion issues
• Conducts trainings and provides technical
  assistance to states, providers, and consumers,
  and
• Provides national database of restraint and
  seclusion incidents, program outcomes, etc.

28
For Further Information

• Lloyd Bullard
• Project Director
• [phone]

• Darren Fulmore
• Evaluation Coordinator
• [phone]

Katie Johnson Research Assistant [phone]
Coordinating Center for Behavior Support and
Intervention behaviormanagementtraining_at_cwla.org o
r http//www.cwla.org/programs/behavior/