Intervention Design and Evaluation

1
Intervention Design and Evaluation

• Celia P. Kaplan, DrPH, MA
• Research with Diverse Communities
• EPI 222, Spring
• June 4, 2009

2
Phases of Disparities Research

• Detecting
• Define health disparities
• Define vulnerable populations

• Understanding
• Identify determinants and mechanisms of
  disparities

• Reducing Health Disparities
• Intervene
• Evaluate
• Translate/disseminate
• Change policy

Adapted from Kilbourne et al., 2006
3
Health Interventions Definition

• A health intervention is a specific activity
  (or set of related activities) intended to bring
  about change in a particular target population
  using a common strategy for delivering the
  messages. CDC
• Example An individual-level counseling
  intervention may consist of four related
  sessions, but they are all provided in a clinic
  through one-on-one interaction.

4
Phases of Research

• Phase I Hypothesis development
• Phase II Methods development
• Phase III Controlled intervention trials
• Phase IV Defined population studies
• Phase V Demonstration and implementation
• Nationwide prevention and health services
  programs

5
Interventions

• Types of interventions
• Lifestyle behaviors
• Smoking, exercise, weight management, alcohol
  use aids prevention, contraceptive use
• Quality of life
• Depression, fatigue, pain, sleep, social support
• Decision making
• Health behaviors
• Cancer screening, medication adherence
• Health Services
• Reminders, flow-charts

6
Interventions

• What makes a successful intervention?
• Efficacious
• Effective
• Exportable
• Efficient
• Relationship of the intervention effect relative
  to the cost

7
The Intervention Cycle
8
The Intervention Cycle
1. Problem Identification
9
Problem Identification

• Methods
• Epidemiology
• Qualitative methodology

10
Five-Year Relative Survival Rates by Race and
Stage United States (SEER), 1975-1979 to 1992-1999
Source Ghafoor, Jemal et al, 2003
11
Five-Year Relative Survival Rates by Race and
Stage United States (SEER), 1975-1979 to 1992-1999
Source Ghafoor, Jemal et al, 2003
12
Use of Mammography in the Last Two Years for
women 40 years of age and over, by race and
Latino origin
Source National Center for Health Statistics,
2005
13
Problem Identification

• Qualitative methodology (focus groups,
  semi-structured interviews, key informants)
• Barriers
• Access to mammography, insurance, transportation,
  lack of referral, language, lack of information
• Facilitators
• Reminder systems, support systems
• Cultural factors
• Family, health orientation
• Individual factors
• Quantitative methodology (surveys)

14
The Intervention Cycle
2. Intervention Design
15
Intervention Design

• Identify strategy and theory
• Develop messages and materials
• Test the intervention

16
Intervention Design

• Strategy
• Selection of the intervention strategy is
  influenced by
• Expertise and interest of investigators
• Resources
• Access to the population
• Distribution of the population
• Access and familiarity of the population with the
  methodology (e.g., Internet)
• Cultural preferences of the population

17
Intervention Design Theory

• Selection of a theory examples
• Health Belief Model (perceived susceptibility,
  severity, benefits, barriers, cues to action,
  self efficacy)
• Transtheoretical Model (precontemplation,
  contemplation, preparation, action, maintenance)
• Precede-Proceed planning model (predisposing,
  reinforcing, enabling factors)

18
Intervention Design

• Theory
• Concerns about the applicability of traditional
  theories to the study of multiethnic populations
• Traditional theories emphasize the role of the
  individual
• Latinos and Asian Americans may prioritize family
  needs over their own

19
Intervention Design

• Culture
• particular groups values, beliefs, norms and
  life practices that are learned, shared, and
  handed down
• Cultural beliefs include
• Etiology of an illness
• The nature of symptoms
• Modality of treatments
• Preventive measures
• Expectations and roles of the ill individual

20
Intervention Design

• Issues to consider
• To what extent is the targeted behavior
  culturally determined?
• What are the most effective routes to reaching
  the population?
• Must the content of the educational message
  address cultural factors?

21
Intervention Design

• Achieved by
• Involvement of community member
• Research conducted by representatives of the
  culture (PIs, Co-Is)
• Extensive qualitative methodology

22
Intervention Design

• Cultural leverage
• A focused strategy for improving the health of
  racial and ethnic communities by using their
  cultural practices, products, philosophies, or
  environments as vehicles that facilitate behavior
  change of patients and practitioners.
• Fisher, et al, 2007 Medical Care Research and
  Review

23
Intervention Design

• Cultural leverage (Fisher et al, 2007)
• Presents health messages in the context of social
  and/or cultural characteristics
• Activates shared norms and expectations
• Addresses the uniqueness of each culture
• Adapts the messages to the shared preferences of
  specific cultural groups
• Makes health care system cognizant of cultural
  practices
• Implements interventions by professionals from
  the targeted groups

24
The Continuum of Culturally-Tailored Interventions
Generic interventions with modifications
Cultural targeting
More Individually Tailored
More Generic
Identification of common elements across cultures
Individual cultural tailoring
25
Intervention Design

• Cultural targeting
• Strategies to try to reach group members who
  share certain values, beliefs, and practices
• Cultural tailoring
• Customizes messages based on an individual status
  (e.g., level of spirituality, collectivism,
  racial pride)

26
Intervention Design

• Development
• Review existing materials
• Example Breast cancer risk assessment programs
• Develop own materials
• Basic components
• Source
• Message
• Channel
• Setting

27
Intervention Design Components

• Source
• Expertise
• Trustworthiness
• Background similarity

28
Intervention Design Components

• Source
• Community leaders
• Community health workers (promotoras)
• Navigators
• Health educators
• Physicians

29
(No Transcript)
30
Intervention Design Components

• Source
• Example Witness Project in rural Arkansas
• Witness role models (cancer survivors) led small
  discussion groups
• Discussion of cancer experiences at churches and
  community settings
• Self-reported breast self examination and
  mammography rates increased significantly from
  baseline to six months
• Erwin et al., 1999

31
Intervention Design Components

• Message
• How the message content and structure
• influence communication effectiveness
• Approaches
• Peripheral. Packages the contents in ways likely
  to appeal to a specific audience
• Colors, images, drawings, pictures

32
Intervention Design Components

• Message
• Evidential Enhances the perceived relevance of a
  health issue to the specific group
• Example Breast cancer mortality is higher for
  African Americans than any other group
• Linguistic strategies Uses the language of the
  audience
• Literacy
• Language preference

33
Intervention Design Components

• Message
• Socio-cultural approaches Presents health
  messages in the context of social and/or cultural
  characteristics
• Appeal
• Positive
• Humorous
• Threat/ fear

34
Intervention Design Components

• Message
• Example
• Population African-American women
• Promotion of mammography
• Compared cultural vs. behavioral tailoring
• Cultural tailoring ( spirituality, collectivism,
  racial pride)
• Behavioral constructs tailoring (knowledge,
  perceived risk, perceived barriers)
• Results Women who received the behavioral
  tailoring were more likely to remember the
  message than those who received the cultural
  tailoring (Kreuter et al, 2004)
• Women receiving BCT CRT magazines were more
  likely than those in the BCT, CRT, and control
  groups to report getting a mammogram (Kreuter et
  al, 2005)

35
Intervention Design Components

• Channel
• The mechanism by which the message is delivered
• Interpersonal (physicians, friends, counselors)
• Group (classroom activities)
• Mass media channels (radio, print media)
• Interactive digital media (Web, Kiosk, games,
  videos, email)

36
Intervention Design Components

• Channel
• Ethnic media addresses relevant issues to the
  communities they serve
• Access to technology varies by ethnic group
• Familiarity with technology
• Perceptions of the Internet may vary by ethnic
  group

37
Intervention Design Components

• Setting
• Places where the intervention can reach the
  intended program
• Home
• School or work
• Community organization
• Support groups
• Times when the audience members may be more
  attentive
• Waiting room
• Places or situations in which they will find the
  message more credible
• Medical centers, churches

38
Intervention Design

• Testing
• Methods
• Pretesting with intended audience members
• Self-administered questionnaires
• Interviewer administered
• Direct observation
• Readability assessment
• Cognitive interviews

39
Intervention Design Testing

• Pretesting
• Assess comprehensibility (language)
• Determine personal relevance
• Identify confusing, sensitive, or controversial
  elements
• Assess attention
• Content of the intervention
• Test the protocols

40
The Intervention Cycle
3. Intervention Implementation
41
Intervention Implementation

• Identification of staff
• Bilingual
• Bicultural
• Training of staff
• Guarantee the fidelity of implementation

42
The Intervention Cycle
4. Evaluate Findings
43
Evaluation

• Answers the question Have the intervention
  objectives been achieved??
• An assessment of the good and bad points of an
  intervention and how it can be improved

44
Evaluation

• Types of evaluation
• Process evaluation
• The process of intervention implementation and
  how the intervention performed as implementation
  takes place.
• Impact evaluation
• Immediate effects of the intervention or
  short-term outcome
• Outcome evaluation
• Outcome evaluation involves an assessment of
  long-term effects of an intervention

45
Evaluation

• Impact evaluation
• Measure changes in health awareness,
• knowledge and attitudes
• Measure interest shown by target groups, e.g.,
  uptake of health education materials, phone-ins,
  participation in screening, etc.
• Observation, questionnaires, interviews,
  discussions, etc.
• Use of attitude scales

46
Evaluation

• Outcome Evaluation
• This is the preferred evaluation method because
  it measures sustained and significant changes
  which have stood the test of time.
• Uses hard evidence and quantitative methods.

47
Evaluation Design

• Gold standard
• Randomized control trial
• Difficulties
• Difficulties to identify the population
• Recruitment
• Contamination at clinical settings
• Other designs
• Controlled before-and-after study
• Time series design

48
Evaluation

• Evaluation of interventions addressing health
  disparities
• a) Interventions using cultural leverage
• Cultural Leverage Interventions using culture to
  narrow racial disparities in health care Fisher
  et al Med Care Res Rev 2007 64 195S
• b) Generic interventions addressing breast cancer
  disparities at the health care setting
• Individual
• Provider
• Interventions to enhance breast cancer screening,
  diagnosis, and treatment among racial and ethnic
  minority. Masi et al., Med Care Res Rev 2007 64
  195S

49
Evaluation

• Evaluation cultural leverage interventions
• Much of the research on cultural targeting and
  cultural tailoring focuses on
• Participants reactions to the interventions
• Less robust evaluation of the outcome of the
  intervention

50
Health Disparities Interventions

• Patient interventions (Masi et al., 2007)
• Patient targeted screening trials
• Reminder letters
• Written educational materials
• Telephone calls
• Culturally tailored classroom instructions
• Videos
• Effect of these interventions was not uniform
  among all groups
• Studies among low-acculturated Latinas
  demonstrated a positive effect of culturally
  tailored interventions

51
Health Disparities Interventions Breast Cancer
Screening

• Provider interventions (Masi et al., 2007)
• Chart reminders
• Chart flow sheets
• Written educational materials
• Chart audits and feedback
• Financial interventions
• Assistance with financial and logistical needs
  increased mammography in patient populations that
  were diverse with respect to race, ethnicity, and
  insurance status
• Dramatic increases associated with vouchers
• Provider interventions led to greater increases
  in screening mammography compared to patient
  targeted interventions

52
Health Disparities Interventions

• Breast cancer interventions
• The majority of these interventions were
  implemented by nurses and lay personnel
• Interventions using culturally specific patient
  navigators and community health workers were the
  most successful strategies
• None of the studies addressed the extent to which
  cultural aspects of the intervention brought
  about the improvements in care

53
Conclusions

• Interventions design and implementation need to
  address cultural as well as individual factors
• Cultural leverage may increase the intervention
  effect
• More research is needed to address how much
  cultural tailoring/targeting is needed
• Tailoring may be necessary to reach those at
  highest risk or those less likely to adopt new
  technologies
• Need to do rigorous studies that assess outcome
  in studies that use cultural leverage