Constructing a Cross-Site Evaluation of Ethnic Minority HIV Mental Health Services The Mental Health HIV Services Collaborative (MHHSC) Program

1
Constructing a Cross-Site Evaluation of Ethnic
Minority HIV Mental Health ServicesThe Mental
Health HIV Services Collaborative (MHHSC) Program

• Michael Costa, Abt
• Barbara J. Silver, CMHS
• Maria Madison, Abt
• Tandiwe Njobe, Abt
• Gabriela Garcia, Abt

2
Presentation Goals

• Describe the MHHSC Program
• Components
• Context
• Relevance
• Convey
• The process of this collaborative.
  utilization-focused evaluation
• Accomplishments to date (products)
• Uses of the outputs of products

3
Substance Abuse Mental Health Services
Administration(SAMHSA)

• Center for Mental Health Services (CMHS)
• Center for Substance Abuse Prevention (CSAP)
• Center for Substance Abuse Treatment (CSAT)

4
SAMHSA HIV/AIDS HISTORY

• Prior to 2001 CSAT funds targeted
  capacity/HIV/AIDS Substance Abuse treatment
  programs for African American, Hispanic/Latino,
  and other racial/ethnic minorities.
• 2001 CMHS funds a similar targeted/expanded
  capacity program for community based
  organizations (CBOs) serving African American,
  Hispanic/Latino, and other racial/ethnic
  minorities.

5
HIV Infection among People with Severe Mental
Illness
Across all published studies, the rate of HIV
infection among psychiatric patients is 10, 25
times higher than that of the general
population. Cournos McKinnon, 1997Krakow et
al., 1998Rosenberg et al., 2001
6
Sexual Risk Behavior Among People With Severe
Mental Illness

COMPARED TO GENERAL POPULATION, PATIENTS HAVE
FEWER EPISODES OF SEX WITH A PARTNER, BUT THEY
HAVE

• ? Number of partners
• ? Number of risky or anonymous partners
• ? Frequency of sex trading
• ? Rates of coerced sex
• McKinnon et.al., 1996, 1999

7
Psychiatric Disorders and Risk for HIV Infection

• Elevated risk for HIV infection in psychiatric
• Patients
• Risk factors
• Alcohol and other drug use
• Unsafe sex
• Environmental circumstances (poverty,
  institutionalization, etc.)
• Substance use is associated with both psychiatric
  symptoms and HIV risk

8
MHHSC Program

• 21 Mental Health Service Sites CBOs
• at least 2 years experience in behavioral health
  care services
• MH Centers, Substance Abuse facilities,
• Primary Health Care /or HIV/AIDS clinics
• Abt Associates, Inc. Coordinating Center

9
MHHSC Program

• Congressional requirement (CBC CHC) provide
  these new HIV/AIDS-related mental health services
  in both traditional and non-traditional settings.
• Funding for mental health treatment services and
  related case management only.
• However, grantees are required to develop
  comprehensive integrated individual treatment
  plans and monitor primary and substance use
  treatment.

10
WHO ARE THE SERVICE SITES?

• New HIV/AIDS-Related Services
• New services (no prior HIV/AIDS-related MH
  services) 5 sites
• Expanded services 16 sites
• Service Delivery Settings
• Traditional (primarily clinic-based) 13 sites
• Non-traditional (e.g., mobile treatment, ) 1
  site
• Both settings 8 sites
• Target Populations
• African American -19 sites
• Hispanic/Latino 14 sites
• Haitian 1 site
• Native American 1 site

11
Demographics Gender
12
Demographics Race / Ethnicity
Hispanic Latino/a 30.65
13
Demographics - Age
14
DSM IV Diagnoses
15
DSM IV Diagnoses Categories
16
MHHSC Program Goals

• Expand
• Effective
• Culturally Competent
• Mental Health Services
• For PLWHIV
• In Minority Communities

17
MHHSC Capacity Building

• The Coordinating Center provides technical
  assistance to grantees
• Local and regional trainings (e.g., ethics,
  neuropsychology, cultural competence)
• Expert speakers at national meetings on topics of
  mental health and psychiatry, gender issues,
  consumer involvement, etc.
• Assistance with data collection and management

18
CULTURAL COMPETENCE

• Strategies to Promote Culturally-Competence
  Service Delivery
• Demographically similar staffing as target
  population(s)
• Treatment facilities reflect cultural interests
  of target group(s) (e.g., artwork/decor, waiting
  room music, etc.)
• Diversity/cultural competence training
• Consumer feedback
• This is being addressed in great detail by the
  MHHSC Cultural Competence Subcommittee

19
MHHSC Utilization-Focused Cross-site Evaluation

• Clinically/Programmatically Relevant Evaluation
• meet the needs of the clinical and other program
  staff in their efforts to better serve their
  clients
• The MHHSC cross-site evaluation is voluntary,
  except for submission of required aggregate data
• critical that the cross-site evaluation be
  clinically and programmatically relevant and
  valuable, otherwise no site would agree to
  participate

20
Evaluation as a Pain in the Neck

• Mental health/social service providers
  historical experience with evaluation
• Intrusive reporting requirement
• Necessary to satisfy Local, State and Federal
  funding requirements.
• Often data are never reported back to programs
• Concerns that data will used to make them look
  bad or draw inappropriate comparisons between
  sites and providers

21
MHHSC X-Site Evaluation Collaborative Process

• Engaging the key stakeholders
• Local site-specific evaluators
• Local site clinicians
• Program Administrators
• Consumer Advisory Boards (CABS)

22
MHHSC X-Site Evaluation Collaborative Process
Evaluation Subcommittee (ESC)
23
MHHSC X-Site Evaluation Collaborative Process
24
X-Site Candidate Evaluation Foci
25
Quantitative Methods Used Across Sites
26
Qualitative Methods Used Across Sites
27
MHHSC X-Site Evaluation Collaborative Process
28
MHHSC X-Site Evaluation Collaborative
ProcessNext Steps
29
Final Cross Site Foci

• Four main domains of interest across sites
• Mental Health
• Client Satisfaction
• Client Retention and Service Utilization
• Quality of Life Medical Health
• Cultural Competence appears in all domains.

30
Common Questions of Interest

• Who is being served?
• What are the barriers to care and to services for
  the target population? How do programs overcome
  these barriers?
• What services are being used by the target
  population?
• Are the services being provided in a culturally
  competent manner?

31
Mental Health

• Given the target population being served, what
  are the
• Prevailing mental health diagnoses?
• Co-occurring disorders? (e.g., substance use)
• Changes in mental health symptoms?
• Disorders that may be more prevalent with HIV
  positive status?
• Physiological HIV disease factors that contribute
  to mental health symptoms?

32
Client Satisfaction

• Questions relating to care received by clients.
• What care is being received?
• What is the frequency of care?
• What is the level of client involvement in care?
• What is the setting in which care is received?
• Is the client satisfied with the care?

33
Client Retention Service Utilization

• Client Retention
• How many cases become active/inactive in a given
  time period?
• What is the site definition for active/ inactive
  clients?
• What are the site policies for case closing?
  What reasons are given for case closings?
• What retention strategies have been effective
  with the target population?

34
Client Retention Service Utilization -
continued.

• Service Utilization
• What is the type, frequency and duration of
  services used?
• How are services provided?
• Referrals in and out of program
• Agency linkages - in-house and external - to
  other providers
• Service setting traditional/ non-traditional

35
Quality of Life

• How does the quality of life status and general
  medical health of a client impact upon
• Treatment compliance (medications, ITP)?
• Response to treatment?
• Client retention?
• Service utilization?
• How does quality of life status and general
  medical health change over time with treatment?

36
Methods Indicators

• CLIENT LEVEL INSTRUMENT
• confidential unique client id

• Demographics
• Social supports
• Acculturation
• Substance use and risk behavior
• Trauma
• Medical health
• Medication adherence
• Clinicians report

37
Methods Indicators

• CLIENT SATISFACTION SURVEY
• anonymous

• Client characteristics
• Service utilization
• Care
• Client involvement in care
• Access/ barriers to care
• Cultural competence in care
• Overall satisfaction with care

38
Methods Indicators

• FOCUS GROUPS
• with clients on site

• Focus groups will provide qualitative backdrop
  to analyze quantitative data from client
  satisfaction survey and client instrument
• Types of services used
• Satisfaction with services
• Barriers to care
• Cultural competency

39
Methods Indicators

• SITE VISIT DATA COLLECTION
• MHHSC Coordinating Center Staff

• Continuum of services
• Location of site
• Geographic setting
• Site community
• Service setting (traditional / non traditional)
• Client retention strategies
• Site activities to overcome identified barriers
  to mental health care

40
Utility of Cross-Site Analyses From the Sites
Perspective

• Site buy-in has been an on-going process
• Clinicians and local evaluators participate in
  the evaluation design work group
• They took ownership of the evaluation design
• Made decisions on utility of collecting certain
  types of data across sites

41
Client Focused Domain

• Client Characteristics
• Clients Presenting Diagnosis
• Barriers/Access to Care (enhance/maintain
  Clients Quality of Life/Health)

42
Program Context Domains

• Program Structure
• CBO VS Large System of care
• Staffing patterns (FT, PT)
• Sustainability Efforts
• Plan in place

43
Client Focused Domains
44
Client Focused Domains (cont.)
45
Client Focused Domains (cont.)
46
Program Context Domains
47
Program Context Domains (cont.)
48
Program Context Domains (cont.)
49
Purposes/Uses of Data

• Cross-site analysis
• Attention to differences across sites, client
  characteristics
• Careful appropriate
• Nuanced, not reductionist
• Program context critical
• Use of qualitative and program-level data (e.g.,
  differences in resources/capacity available)

50
Purposes/Uses of Data (Recap)

• UTILIZATION-FOCUSED EVALUATION
• Data/analysis to be provided to sites
• User friendly feedback
• A timely manner
• For program improvement /development of better
  intervention strategies
• NOT REPORT CARDS
• Individual clinicians
• Individual sites