Towards a ConsumerSurvivor Research Agenda

1
Towards a Consumer/Survivor Research Agenda
Consumer Operated Service Program (COSP)

• Jean Campbell, Ph.D. Jeanne Dumont, Ph.D.
• The 26th Congress of the
• World Federation for Mental Health
• Vancouver, British Columbia, Canada
• July 22-27th, 2001

2
Consumer-Operated Service Program
Research ought to and can enhance consumer
choice, power, and knowledge
From Consumer/Survivor Mental Health
Research Policy Work Group (1993)
3
Outline

• Introduction
• The Well-Being Project
• The Peer Outcomes Protocol Project
• The Crisis Hostel Project
• The Consumer-Operated Service Program Multi-site
  Research Initiative
• Conclusion

4
Self-Help Research

• Numerous research studies have shown that
  participation in self-help groups can help people
  improve the quality of their lives significantly.
• Studies have found that support groups can reduce
  the need for medical care and hospitalization.

5
Consumer-operated service programs emerged in the
1980s as an alternative to traditional mental
health services.
6
Research on Consumer Programs

• Found a decline in both symptoms and concomitant
  psychiatric treatment as a result of
  participation in consumer-operated services
  (Galanter, 1988).
• Positive changes in perception of self, social
  functioning, decision-making, and symptomatology
  linked to participation in consumer-operated
  services (Carpinello et al., 1992).

7

• In 1979 Prager and Tanaka reported to the Ohio
  Department of Mental Health on the results of
  involving mental health consumers in evaluation.
  They concluded Representing the consumers
  perspective on the meaning of mental illness and
  the correlates of getting better, the process
  of client involvement in evaluation design and
  implementation is not only realistic and
  feasible it is, we feel, a professional
  necessity whose time is overdue.

8
Towards a Consumer/Survivor Research Agenda

• The Well-Being Project
• Mental Health Clients Speak for Themselves
• (1986-1989)

9
A Landmark Study

• Funded by the California Department of Mental
  Health Office of Prevention
• First consumer research project Conducted by the
  California Network of Mental Health Clients
• Jean Campbell, Principal Investigator
• Ron Schraiber, Co-Investigator

10
Research Question

• What promotes or deters the well-being of adults
  with severe and persistent mental illness in
  California?

11
Study Design Developed, Administered and
Analyzed by Mental Health Consumers

• State-wide Survey
• Qualitative and Quantitative Methods
• focus groups to develop items
• open-ended questions included
• in-depth interviews
• multiple choice, likert scaled items

12

• Triangulation
• clients (N331)
• family members (N53)
• mental health professionals (N150)
• Convenient sample
• Consumer surveyors
• face-to-face interviews
• self-administered interviews (mail)
• group interviews

13
Discovering the Consumer Perspective

• Asking New Questions
• negative outcomes identified
• self-management skills revealed
• importance of personhood established
• effects of prejudice and discrimination
  quantified
• incongruity of values, perspective, and
  identified needs between consumers, family
  members, and mental health professionals found

14
Findings

• 40 of mental health clients surveyed felt that
  all or most of the time people treated them
  differently when they found out they have
  received mental health services
• like they are violent (16)
• like a child (21)
• like they dont know what is in their own best
  interest (31)
• like they are in capable of caring for children
  (20)
• like they are incapable of holding a job (33)

15
Findings

• More than half of the clients surveyed indicated
  that they always or most of the time
  recognize signs or symptoms of psychiatric
  problems and that they can take care of these
  problems before they become severe.

16
Findings

• 48 indicated that they have avoided treatment
  due to fear of involuntary commitment.
• 30 reported that they had little or no
  control over the kind of mental health services
  they receive.

17
Peer Support Outcomes Protocol

• Development of an Evaluation Protocol for
  Community-Based Peer Support Programs
• (1996-2001)

18
To survive in an era of evidence-based
funding, peer support programs need to measure
cost, effectiveness, quality, utilization
and appropriateness of the services they provide.
19
Purpose of the POPP

• The Peer Support Outcomes Protocol Project
  developed, field-tested, and will soon distribute
  an evaluation protocol that measures outcomes and
  satisfaction of community-based peer support
  programs that are operated by mental health
  consumers/survivors.

20
Consumer-Developed Project

• Protocol developed and tested by the Program in
  Consumer Studies and Training at the Missouri
  Institute of Mental Health in St. Louis.
• Funded by the National Research and Training
  Center on Psychiatric Disability at the
  University of Illinois-Chicago

21
Outcome DomainsSpecific outcome domains
organized into individual modules

• Demographics Services
  Hospitalizations
• Employment Housing/Community Life
• Social Support Quality of Life
• Well-Being (Recovery, Empowerment Personhood)
• Crime/Violence Program Satisfaction

22
Use of the POPP

• Assist the consumer self-help field to assess
  program outcomes
• Present service outcomes to public funding
  authorities and manage-care organizations
• Help consumers improve the organization and
  delivery of peer support programs

23
Phase One (1996-1997)

• National Survey of Data Needs of Peer Support
  Programs (N30)
• Sorting and ranking of survey items from review
  of consumer literature and instruments
• Instrument development
• IRB approval

24
Phase Two (1997-1998)

• Protocol Field-testing
• Instrument review and refinement
• organization
• clarity of item wording
• respondent burden
• ease of administration
• consumer sensitivity

25
Phase Three (1998-2000)

• Development of Interviewer Training Manual
• Development of Q-by-Q
• Psychometric Testing (test N150 retest48)
• reliability
• validity

26
Phase Four (2000-2001)

• Tool Kit Development
• Knowledge Exchange
• Plans for APS (application program services)

27
Towards a Consumer/Survivor Research Agenda

• Consumer-Operated Service Program
• Multi-Site Research Initiative
• (1998-2002)

28
Consumer-Operated Service Program
The Consumer-Operated Service Program
(COSP) Multi-site Research Initiative is a
federally-funded national effort to discover to
what extent consumer- operated programs as an
adjunct to traditional mental health
services are effective in improving the
outcomes of adults with serious mental illness.
29
Features of COSP

• Multi-site Study
• Seven study sites located throughout
• the U.S.
• Coordinating Center in Missouri
• Four years of research supported
• 20 million dollars in research funds
• Randomized experimental design

30
Consumer-Operated Service Program
Federal Representatives Substance Abuse and
Mental Health Services Administration
(SAMHSA) Center for Mental Health Services
(CMHS) Betsy McDonel, Ph.D., federal program
director Crystal Blyler, Ph.D., government
project officer

• Coordinating Center
• Missouri Institute of Mental Health
• Jean Campbell, Ph.D., principal investigator
• University of Massachusetts Medical Center
• Matt Johnsen, Ph.D., co-principal investigator

31
Participating Study Sites

• Maine
• Missouri
• Pennsylvania
• Tennessee

• Connecticut
• Florida/California
• Illinois

32
Participating Study Sites
33
What is a COSP?

• A consumer-operated service is
  administratively controlled and operated by
  consumers and emphasizes self-help as its
  operational approach

34
Consumer-Operated Program Models

• Drop-in Centers
• Educational Advocacy Training Programs
• Peer or Mutual Support Services

35
COSP Goals

• Establish the extent to which consumer-operated
  services are effective in improving selected
  outcomes for consumers of mental health services
• Create strong and productive partnerships among
  consumers, service providers and service
  researchers
• Disseminate the knowledge gained

36
Target Population

• Study participants are defined as persons age
  18 and over who currently or at any time over the
  past year have had a diagnosable mental,
  behavioral, or emotional disorder of sufficient
  duration to meet diagnostic criteria specified
  within the DSM-IV that has resulted in functional
  impairment which substantially interferes with or
  limits one or more major
• life activities.

37
COSP Big Six Outcomes

• Employment
• Empowerment
• Housing
• Service Satisfaction
• Social Inclusion
• Costs

38
Cost Study Questions

• To what extent does participation in
  consumer-operated service programs affect costs
  for the following
• inpatient
• crisis intervention
• emergency room utilization
• offsetting costs in housing, criminal justice,
  vocational rehabilitation, physical health care,
  and income support

39
Research Design
Rigorous Methodology

• Multi-site Design
• Random Assignment
• Experimental

Intervention Consumer-Operated Program
Traditional Mental Health
Services Control Traditional Mental Health
Services Only
40
Research Design

• Logic Model
• Common Protocol
• Data Collection
• Baseline, 4, 8, 12 months
• N2,230

41
Multi-level, Cross-site Analysis Plan

• Data quality assessment development of measures
• development of composite scores scales
• missing values analysis
• equivalencies
• power effect size
• Baseline Data Analysis
• measures of central tendency dispersion
• factor analysis
• correlations
• study population description

42
Multi-level, Cross-site Analysis Plan

• Data Clustering Pooling
• Longitudinal Analyses
• hierarchial linear modeling
• pattern-mixture modeling
• differential loss to follow-up
• random regression

43
Measuring Fidelity/Implementation

• Fidelity/Implementation Assessment
• Site Visits
• Common Ingredients (CI)
• Service Matrix
• Service Integration Assessment
• Fidelity Assessment Common Ingredients Tool
  (FACIT)

44
Common Ingredients

• Structure
• Consumer-Operated
• Participant responsive
• Links to other supports
• Environment
• Accessibility
• Safety
• Informal Setting
• Reasonable accommodation

• Belief Systems
• Peer principal
• Helpers principle
• Empowerment
• Choice
• Recovery
• Acceptance and Respect for
  diversity
• Spiritual growth

45
Common Ingredients

• Education
• Self-management/problem solving
• Education
• Advocacy
• Self-advocacy
• Peer advocacy
• Systems advocacy

• Peer Support
• Peer support
• Telling out stories
• Consciousness-raising
• Crisis Prevention
• Peer mentoring and teaching

46
Multisite Data Repository

• Coordinating Center responsibility
• Data electronically transferred via the internet
  from 8 sites across the country
• Multiple data points in one database
• Over 1700 data elements with 12,566,400
  observations
• Data consistency capabilities through queries

47
Data Repository Specifications

• Database
• Microsoft Access 97 (version 8)
• self documented
• secured (user ID and password)
• encrypted
• Backups
• backup plan required from sites
• daily, monthly rolling backups
• quarterly permanent backups

48
Data Trail
Common Protocol
MIMH
Files from 7 Sites
Data Entry
Merged Files
Multiple Files
Transmitted to CC
Data Analyzed
Merged Files
ROW
49
Data Entry Program

• Created to allow data entry at the site level
• Missing variables not allowed
• Programmed to allow only in-range responses
• Designed for heads down data entry
• Automated skip patterns
• Required double entry verification

50
Data Entry Screen
51
(No Transcript)
52
Study Progress Monitoring

• Site Reports to Coordinating Center
• Weekly Flash Report
• Track enrollment by condition
• Quarterly Report
• Report on recruitment, enrollment, and attrition
• Report cost data status
• Narrative on site accomplishments, problems
  encountered, and goals

53
Study Progress Monitoring

• Coordinating Center Reports
• Enrollment
• Engagement
• Cost data
• Projections
• Power analysis
• Baseline equivalencies
• Other summaries

54
Quality Control Functions

• On-going monitoring of standardized interviewing
• Interviewer Alerts
• Updates to the Question by Question Manual
• Follow-up windows monitored
• Patterns of missing variables tracked
• Collection of diagnoses monitored

55
Key Operational Values

• Consumer Involvement
• Consumer Education
• Extensive Technical Assistance
• Electronic and Interactive Communications
• Collaboration
• Cultural Competency

56
Building Partnerships

• Can consumers, service providers, and
  researchers
• complement each others strengths?
• Do joint efforts yield the most effective
  service
• delivery model possible?
• Will consumer-operated services have a crucial
  role to
• play in mental health service delivery system?

57
COSP Basic Principles
58
Consumer Collaboration

• SC Consumer Involvement
• SC Consumer Representatives
• Consumer Advisory Panel
• Study Sites Consumer Involvement
• Site Consumer Advisory Boards
• Consumer Researchers
• CC Consumer Research Support
• Research Glossary
• Workshops
• Technical Assistance

59
Building Trust

• Policies to ensure access to project information
• Supportive communications infrastructure
• Defined decision-making process
• Telling our stories
• Use of the language we, our and us
• Voting, focus groups, and ad hoc subcommittee
  meetings

60
Building a Learning Community
Learning is the core objective and
this should guide decision- making. Mike
English, CMHS
61
Consumer-Operated Service Program
One of the hallmarks of the COSP is the effort
made to use technology to facilitate work and
disseminate information. Visit our
website http//www.cstprogram.org
Consumer Operated Service Program (COSP)
62
CONCLUSIONIntegrating Diverse Cultures into the
Conduct of Research

• The failure to include consumers and other
  culturally diverse groups within services
• research may
• compromise research findings
• hinder the ability of service providers and
  policy-makers to understand consumer needs.

63
Nothing About Me, Without Me

• Adopting the slogan Nothing about me, without
  me, mental health consumers and people of color
  have moved rapidly to be involved in the design
  and implementation of mental health services
  research and evaluation.

64

• The growth and acceptance of such partnerships
  show the potential for progress when different
  cultures work together in relationships of
  mutuality and respect.
• However, the inclusion of diverse groups within
  the conduct of research has presented interesting
  challenges that expert-driven models of
  research have proven inadequate to address.

65

• In particular, the empowerment of mental health
  consumers in the administration, design,
  implementation, and analysis activities has
  necessitated an on-going dialogue between
  consumers, consumer researchers, and non-consumer
  researchers to reach common ground regarding
  issues of
• authority,
• expertise,
• and language.

66
What Divides Us

• It is important to remember that mental health
  programs, including those that are
  consumer-operated, function within a political
  system in which data are often exercised in
  struggles for influence.

67
No Common Language

• There is no common language or experiences that
  would naturally bring different constituencies
  together.

68

• Values and goals that arise from culturally
  dissimilar experiences, tend to separate people
  and polarize discussion.

69
Bridging Differences

• Bridging differences between people on a personal
  level needs to be supported through group
  activities that promote respect, understanding,
  and appreciation of the difficulties that
  collaboration presents.

70
What Is Necessary

• In order to accommodate the diverse cultures that
  are now part of the research environment, it is
  necessary to pause and encourage critical
  discourse, and to incubate new relationships and
  ideas as participatory processes are established.

71
Looking Within

• Consumer/survivor research has offered the
  opportunity to consumers and professionals to
  look within, and to re-search in a literal
  sense the terrain of a priori assumptions about
  how research should be conducted and by whom.

72
The Need for Rigor

• Scientific rigor in methods and practices must be
  maintained since the weight of disbelief in
  public policy will surely demand that researchers
  push harder for clarity in research designs and
  data quality.

73
Meaningful Research

• Ultimately, consumer participation in research
  will test the proposition that the integration of
  diverse cultures into the conduct of research
  ought to and can enhance scientific knowledge
  that is useful and meaningful for all participant
  groups.