Title: A Mixed Methods Approach to Assessing How System Change Impacts Clinics and Consumers in Public Ment
1 A Mixed Methods Approach to Assessing How
System Change Impacts Clinics and Consumers in
Public Mental Health
- Joel T. Braslow, M.D., Ph.D.
- Semel Neuropsychiatric Institute
- UCLA
- John S. Brekke, Ph.D.
- Frances Larson Professor of Social Work Research
- School of Social Work
- University of Southern California
- Kathleen Daly, M.D., MPH
- Los Angeles County Department of Mental Health
- Mapping Progress in Mental Health Disparities,
May 20, 2009
2The California MHS Act Impact on Practice
Organizational Culture in Public ClinicsNIMH
R-01 MH08067 (Joint PIs Braslow, Brekke Co-PI
Kathleen Daly)
3Study Goals
- What changes occur as a consequence of the MHSA?
- Why does change occur (or not)?
4MHSA in Historical Perspective
- Apparent discontinuity with the past
- Continuities with the past
- Intended and unintended consequences
5Psychiatric Interventions from the 1900s to 1954
Insulin Coma
6Average Daily Antipsychotic Drug Dosage,
California State Hospitals, 1955-2001
7(No Transcript)
8Continuities
"An alternative approach which would call for a
courageous scrapping of traditional concepts and
a gearing of our institutional program to the
concept of therapy rather than of
custody." California Department of Mental Health,
1949
9Two Centuries of Recovery
10UNINTENDED CONSEQUENCES OF POLICIES
11California State Hospital Population, 1850-1995
12- Elpers JR (1989), Public Mental Health Funding in
California, 1959 to 1989, Psychiatric Services
13(No Transcript)
14- Will the MHSA actually transform care in Los
Angeles County? - How do we make sense of the continuities with the
past? - How do we assess both intended and unintended
consequences?
15California and Los Angeles
- California population of 36.8 million
- 12 million live in households below 200 of the
FPL - 8.8 of these low-income residents meet the MHSA
funding allocation criteria of a serious
emotional disturbance (SED) or serious mental
illness (SMI) - Los Angeles County population of 10.25 million
- 3.95 million individuals living in households
below 200 of the FPL - 8.7 of these low-income residents meet the MHSA
funding allocation criteria of a SED - LAC will receive 30 of MHSA funds
16(No Transcript)
17The California MHS Act Impact on Practice
Organizational Culture in Public ClinicsNIMH
R-01 MH08067 (Joint PIs Braslow, Brekke Co-PI
Kathleen Daly)
18Senior Study Team
- Joel Braslow, MD/PhD Psychiatry/History
- John Brekke, PhD Social Work
- Kathleen Daly, MD/MPH Psychiatry
- Beth Bromley, MD/PhD Psychiatry/ Anthropology
- Project Director, Amanda Nelligan, MA Psychology
19Research Staff
- 4 PhD-level Ethnographers
- 4 Consumer Specialists
- 2 PhD Graduate Students
20Collaborations
- Public-Academic Partnership
- LAC DMH
- UCLA
- USC
- Interdisciplinary Collaborations
- Health Services Research
- Health economics
- Biostatistics
- Anthropology
- Ethnocultural Mental Health
21Specific Aims
- Aim 1 County level Assess how LAC DMH develops
and implements MHSA clinical policy over time
examining fiscal, clinical, political, historical
determinants. - Aim 2 Clinic Level Assess how clinics implement
LAC DMHs interpretation of the MHSA, and how
this transforms clinical culture, structure, and
providers understandings of illness and
treatment. - Aim 3 Client Level Assess the impact of local
clinical transformations on clients objective
outcomes and subjective experiences of their
mental health treatment.
22Mixed-Methods Approach
- Goal
- What effects the MHSA has on clients
- Why the observed effects occurred
- Quantitative methods using
- Provider questionnaires
- Client questionnaires
- LAC DMH IS Database
- Qualitative methods using
- Semi-structured interviews
- Ethnographic observations
- Focus groups
23(No Transcript)
24(No Transcript)
25(No Transcript)
26Summary
- The four studies targeted system factors and then
examined their impact on client-level outcomes. - None of them examined clinic level factors, yet
all involved a clinic level context of service
delivery. - The clinic and its context was a black box in
these studies, despite being critical in
mediating between system-level and client level
outcomes.
27Aim 1 Research Questions(Assess how the LAC DMH
develops and implements MHSA clinical policy over
time.)
- What clinical, fiscal, and political concerns
enter into LAC DMH MHSA policy implementation and
policy change over time? - What ways in which past policy decisions (e.g.,
AB 2034) shape current policy decisions?
28Aim 1 Design
- Administrative/historical documents
- Semi-structured interviews (n18)
- three levels (executive management, district
chiefs and staff) - baseline and at the end of years 1, 2, 3
- Five focus groups
- (1) Union of American Physicians and Dentists
- (2) Services Employee International Union (SEIU
local 660) - (3) National Alliance for the Mentally Ill (Los
Angeles Chapter) - (4) Los Angeles County Client Coalition and
- (5) California Mental Health Directors Association
29Remember Aim 2 concerns the clinic Aim 3
concerns the client
30Aim 2
- Clinic Level Assess how clinics implement LAC
DMHs interpretation of the MHSA, and how this
transforms clinical culture, structure, and
providers understandings of illness and
treatment.
31Aim 2 Research Questions
- How and why do providers beliefs, attitudes and
practices change over time? - Do FSP (MHSA) providers differ in their recovery
orientation and competencies relative to non-FSP
providers? - Does MHSA clinic funding affect recovery
competencies/orientation for providers not in
FSPs?
32Los Angeles County
- 469.1 square miles
- 8 Service Planning Areas
- 321 clinics
- 33 directly operated by DMH and treat
approximately 60 of unique DMH adult
outpatients - 288 contract clinics
33Design For Aims 2 3
- Intensive Case Study with Quasi-Experimental
Cross-site Comparisons - 3 MHSA Clinics (2 DO, 1 Contact)
- 2 Non-MHSA Clinics (1 DO, 1 Contract)
34Clinic Site Selection
35Collaborative Sites
- 2 Contract Clinics
- 1 with MHSA funding with Full Service Partnership
- 1 with no MHSA Funding
- 3 Directly-Operated Clinics
- 2 with MHSA funding with Full Service Partnership
- 1 with no MHSA Funding
36Aim 2 Methods
- Ethnographic observation
- Semi-structured provider interviews (n50 6
waves) - Observation of clinician-client visits
- Provider self-administered surveys (n377 3
waves) - Administrative data from LAC DMH IS database
37Provider Assessments
- Survey
- Universal Sample of All clinicians
- Frequency every 12 months
- Semi-structured interviews
- Sample based on purposive criteria
- Frequency 6 months
38Provider Survey
- Recovery Self-Assessment (RSA)-Provider Version
- Gauges providers perceptions of how well
programs implement practices consistent with
principles of recovery. - Attitudes Towards Illness
- Beliefs about A) impact of illness, B)
chronicity, C) client understanding and control
of illness, D) causes of illness, E) impact of
treatment - Clinic Culture
- Clinic culture, structure, climate, work
attitudes (Glisson)
39Provider Semi-Structured Interviews
- Open-ended interviews focusing on beliefs about
recovery, illness and therapeutics - Frequency 6 months
40Provider snapshots of clinic life
- 15 minute face-to-face or phone interactions
every two weeks. Cohort followed over time.
41Aim 3
- Client Level Assess the impact of local
clinical transformations on clients objective
outcomes and subjective experiences of their
mental health treatment.
42Aim 3 Research Questions
- How and why do clients experiences of care
change? - How effective are FSPs relative to usual care in
MHSA-funded clinics? - Does MHSA funding result in worse outcomes for
usual-care clients in MHSA- funded clinics?
43Aim 3 Methods
- Ethnographic observation
- Semi-structured client interviews (n50 6 waves)
- Client self-administered surveys (n616 every
six months over 3 years) - Administrative data from LAC DMH IS system
44Client Sampling Design
45Matching Criteria
- high service utilization or homeless or jail
- diagnosis
- age
- gender
- ethnicity
46Client Assessment
- Client assessment surveys
- Semi-structured interviews
- Observation of client-provider interactions
47Client Assessment Continued
- The BASIS-32 is a brief, widely-used behavioral
health assessment instrument that provides a
comprehensive measure of functioning. - Recovery Self-Assessment Person in Recovery
version gauges perceptions of how well programs
implement practices consistent with principles of
recovery - Working Alliance Inventory Attitudes Towards
Illness Living situation and work functioning - Quality of life measures SWL, AQOL,
Acculturation and Ethnic identity - Frequency 6 months
48Semi-Structured Interviews
- 5 Clients per cell (10-15/clinic)
- Frequency 6 months
- Aim probe clients subjective experiences of
treatment (its process and impact on them) and of
the clinic.
49Overview of Conceptual Domains in Quantitative (
Qualitative) Measures for MHSA Study1.
Organizational and Social Context (staff only)
A). structure is the distribution of
power and formalization of roles (i)
centralization of authority and decision making
in a few people (ii) formalization is the extent
to which rules and regulations guide all work
activities B). culture is the values and
norms that drive behavior (i) constructive,
(ii) passive defensive, (iii) active defensive
C). climate is the psychological impact of the
work environment on the individual (i)
engagement, (ii) stress, (iii) psychological
safety D). work attitudes are the
individuals morale (i) job satisfaction, (ii)
commitment to organization
502. Recovery Assessment (staff and client)
A). Life goals extent to which staff help
with pursuit of life goals B). Involvement
consumer involvement with services and
administrative aspects of clinic C).
Diversity of treatment options linkages to peer
providers, variety of treatment options and
non-mental health activities D). Choice
consumer choice is respected, access to treatment
records, non-coercive treatment methods used
E). Individually-tailored services services
tailored to consumer needs, interests and
ethnicity 3. Attitudes Towards Illness
(staff and client) Beliefs about A) impact
of illness, B) chronicity, C) client
understanding and control of illness, D) causes
of illness, E) impact of treatment
514. Client Functioning (client only) Assesses
client difficulty with A) daily living/role
functioning B) relationship to self and others
C) depression/anxiety D) impulsive/addictive
behavior E) psychosis. 5. Satisfaction with
Life (client only) A) work B) social
relationships C) living situation D)
psychological condition E) self and present
life 6. Alliance with Service Provider (client
only) A) emotional alliance with provider B)
task and goal alliance with provider
52How are we mixing our methods?
- By type quantitative measures, focus groups,
ethnographic observation, semi-structured
interviews - By utilization a) quantitative results are used
to select Ss for qualitative interviews b)
qualitative data will be used to interpret pooled
and site-specific findings (confirmations and
disconfirmations) c) qualitative data will be
used to set context for quantitative findings
(e.g., leadership, relationship with County)
- By linking constructs especially around a)
beliefs and behaviors concerning illness and
treatment b)
53Data collection
- Clients enrolled 454
- Providers enrolled 293
- All surveyed
- 99 semi-structured interviews
- Clients enrolled 454
- All surveyed
- 68 semi-structured interviews
54(No Transcript)
55(No Transcript)
56Preliminary data on usual care clients living
situation (site a)
57(No Transcript)
58(No Transcript)
59(No Transcript)