The Role of Client Outcomes in Evaluating a System of Care Model of Mental Health Delivery

1
The Role of Client Outcomes in Evaluating a
System of Care Model of Mental Health Delivery

• Richard Phillips, Ph.D.
• Director of Research and Evaluation,
• Idaho Child Welfare Research Training Center
• Eastern Washington University
• Patty Gregory, LCSW
• Director
• Idaho Child Welfare Research Training Center
• Eastern Washington University

2
Design

• On site meeting at each demonstration site for
• information gathering
• case staffing observation
• training on evaluation issues
• conducting focus groups for process data
  collection
• Goal setting process
• Council member identification of barriers
  encountered in reaching the goals.
• Critical case review

3
Role of Outcomes in a System of Care

• What are outcomes?
• Changes in adaptive behaviors
• Basic assumption what people do as a result of
  professional interaction is the best measure of
  professional effectiveness
• Whats the difference between outcomes and
  something else?
• Outputs services/activities provided to clients
  to help (them) improve their condition
• Basic assumption the amount of services is the
  best measure of professional effectiveness.

4
Kinds of Outcomes

• Representative Outcomes
• Refers to the health or capacity of a system
  around specific issues or mandates
• Represents how an entity or person is doing on a
  global scale.
• Refers to how individual outcomes relate to a
  larger system.
• Serves to help an entity or person assess how
  they are doing on a global scale.
• Examples include
• State average scores on standardized academic
  achievement tests
• Decreases in CPS referrals or re-referrals
• Decreases in CAFAS scores

5
Kinds of Outcomes

• Functional Outcomes
• Refers to adaptive behaviors of a person or a
  program or system as they strive to reach their
  goals what people actually do to reach their
  goals.
• Serves to enhance the self-reliance of persons in
  relation to the goals in question
• Example
• Client C visited 3 agencies for the first time
  and received food and clothing for a week
• Social Worker B collaborated with the school
  counselor to create an early referral form
• Student K stayed in school and completed his past
  assignments

6
Participatory Evaluation Using Client Progress
to Help Clients Progress

• Functional Outcomes
• What people do as a result of participating in a
  relationship
• Take actions
• Change attitudes
• Increase performance
• Become more able

• Representative Outcomes
• What happens as a result of aggregated functional
  outcomes
• Increased classroom performance
• Increased grades or attendance
• Positive changes in CAFAS scores

7
System of Care for Children

• Set of shared values and principles
• Focus on strengths of family not on diagnosis of
  the child
• Maintain child in their own home, in their own
  school, and in their own community
• Service array planned with family to meet family
  identified needs
• Non-traditional
• Traditional

8
Post Treatment Data

• School attendance/conduct for first full quarter
  after completion of service
• of failing school academic core subject grades
  out of total possible
• CAFAS score
• Post Juvenile Probation Assessment score
• Disposition of case
• Approximate of hours spent by council in
  chambers
• Approximate of hours spent by council members
  in case management
• Number of council members involved in direct
  service on case
• Family satisfaction survey results

9
Nothing About Us Without UsParent perspective on
functional outcomes

• First and foremost, we seek safety
• That our children are insured a quality of life
  as happy productive members of community as
  measured by
• sharing a ceremony or celebration,
• being in this together
• making it through a school suspension while
  adjusting to a new medication
• We need strengths acknowledged, resources valued
  and our children treated with love and respect
• CrossBear, S. (Spring/Summer 2002) Nothing about
  us without us, Data Matters, National Technical
  Assistance Center for Childrens Mental Health,
  Georgetown University Child Development Center

10
Generalized Service Model
Needs Assessment Diagnosis
Client Presenting Issues
Tx plan and Clinical goals
Services Provided
Periodic Assessment Service Plan Review
Case Closure when Clinical goals met
11
Developing Community Council Model
Council Goals
Family Centered Problem Solving
Client Referral
Tx Goals Service Plan
Interventions Assessments
Collaborative Reviews
Case Closure when Council and Family Goals met
12
Conclusion

• When the probable results of inaction, and the
  lack of results of current actions, form the most
  important reasons for referral to a program, then
  a conclusion about standardized data can be made.
• Any data that are used in predictive sense to
  direct what the parent should do are data that
  are seen as harmful to the process of getting the
  family to engage as partners with the council in
  setting out a treatment plan for their child.

13
The Reality of Collaboration

• Council members defensive because of agency
  history
• Lack of experience with engaging families in a
  process where the family leads interventions
• Lack of understanding of the participation
  agencies regulations and guidelines

• Lack of trust in the family and in the
  professional process
• Inability to get key partners to the table not
  full participation
• Lack of available concrete services/programs
• Funding constraints

14
The Role of Advocacy

• A key measure of council effectiveness is the
  willingness and ability of council members to
  integrate council client concerns into their
  daily agency routines
• Advocacy looks like
• making connections with other professionals as
  council members go about their daily routines
• Intervening in ones own agency to help clients
  gain access to services
• Being supportive by other council members in
  challenging status quo treatment options
• Advocacy does not look like being held
  accountable for collecting data to show client
  progress

15
Greatest Challenge Supporting care providers as
they move from an expert, service provider
orientation to a facilitating, partnering
orientation
16
Recommendations

• Ensure that system of care extends into the
  bureaucracy
• Work with local sites to establish data
  collection routines that celebrate client actions
• Provide evaluation support for local councils so
  volunteer hours are spent in collaborative
  functions, not recording or documentation
  functions
• Train to a participatory evaluation model that
  recognizes that client ownership of outcomes may
  be the single most important function of a mental
  health intervention
• Learn to acknowledge that validity and
  reliability may be obstructionist constructs for
  support changes in client behavior

17
Functional Outcome Evaluation

• Client Progress
• Professional Client set
• Goals Goals

18
Bibliography

• Burns, B., Goldman, S., (1998) Promising
  practices in wraparound for children with serious
  emotional disturbance and their families.
  Washington DC Georgetown University Child
  Development Center, CASSP technical Assistance
  Center. Substance Abuse and Mental Health
  Services Administration. U.S. Department of
  Health and Human Services
• CrossBear, S. (Spring/Summer 2002) Nothing about
  us without us, Data Matters, National Technical
  Assistance Center for Childrens Mental Health,
  Georgetown University Child Development Center
• Friesen, B., Huff, B., (1996) Family
  perspectives on systems of care. In B. Stroul
  (Ed.) Childrens Mental Health. Creating Systems
  of Care in A Changing Society. Baltimore, MD
  Brookes Publishing.
• Lourie, I.S., Davis, C. (1999) A Needs
  Assessment of Idahos Children With Serious
  Emotional Disturbances and Their Families.
• Pines, S. (2002) Building Systems of Care A
  Primer. National Technology Assistance Center for
  Childrens Mental Health
• State of Idaho (2002). Building on Each Others
  Strengths. Child Mental Health Initiative Grant,
  GFA SM-02-002 CFDA 93.104.