Blending the Theoretical and the Empirical in Evaluation Science: A Case Example from Cancer Control

1
Blending the Theoretical and the Empirical in
Evaluation Science A Case Example from Cancer
Control/Prevention and a General Discussion

• Ross Conner
• University of California Irvine USA and
• Senior Adviser and Past President International
  Organisation for Cooperation in Evaluation IOCE

2
Workshop Outline

• Evaluation Overview
• The Theoretical Component
• The Empirical Component
• Case Example Chinese-Korean Cancer
  Prevention/Control Program
• Conclusions
• Questions Answers General Discussion

3
1. Evaluation Overview

• Inputs ? Processes ? Outputs ? Outcomes ?
  Impacts
• A simple example program a mathematics class
• Inputs students, teacher, lesson texts, paper
  and pencils, classroom setting
• Processes lessons and exercises taught by
  teacher
• Outputs exercises completed by students
• Outcomes (short-term) Students learn new
  concepts, have new understandings (measures
  tests)
• Impacts (long-term) Students apply new concepts
  and understandings, students advance to next
  level of learning
• For simplicity, this is set out as a linear track
  of 5 stages in practice, it is often more
  complex with multiple tracks because of different
  components of the program.

4
2. The Theoretical Component

• Theory from Evaluation Science
• Evaluation Science bringing the tools of science
  and discovery to the assessment and evaluation of
  programs or policies.
• Tracking and monitoring inputs and processes
  identifying and documenting outputs measuring
  outcomes tracking and measuring impacts.
• Theory from the Program or Policy Area
• Program or Policy Area bringing the past
  learning from the area to advance the area
• Building upon past experimental and theoretical
  work in the area, developing program components
  based on past research and practice.
• Using both types of theories, the program
  components and the evaluation components are
  developed.

5
3. The Empirical Component

• Program Design
• Specify planned inputs and planned processes
• Specify expected outputs, outcomes and impacts
• Specify evaluation questions
• Evaluation Design
• Specify evaluation design(s) to answer the
  evaluation questions. Design the plan for data
  collection
• Specify evaluation measures to obtain necessary
  data

6
4. Case Example

• Area Health cancer control and prevention
• Populations Chinese and Koreans in California,
  USA
• Program focus womens cancers initially, later
  mens cancers
• Program aim disease destigmatization

7
Overview Chinese Program Components

• Materials in Chinese language
• Large free luncheons with expert lecturers
• Free screenings for cancers
• Cancer survivors involved in the programs
• High visibility in community

8
Overview Korean Program Components

• Materials in Korean language
• Used networks of Korean Christian ministers to
  destigmatize cancer survivors featured
• Free screenings for cancers
• Prominent role at public events and fairs

9
The Program

• Inputs ? Processes ? Outputs ? Outcomes ?
  Impacts
• Planned Inputs materials in native language,
  large gatherings of community members, religious
  leaders (Korean), cancer experts (Chinese)
• Planned processes dissemination of materials and
  information to 4,000 people, cancer screenings,
  discussions among community members
• Planned outputs increased knowledge about
  cancer, increased understanding of consequences
  of disease stigmatization
• Expected outcomes increased (by 10) cancer
  screenings, decreased disease stigmatization
• Expected Impacts 100 community cancer survivors
  serving as volunteers and outreach workers for
  the program

10
The Evaluation

• Inputs ? Processes ? Outputs ? Outcomes ?
  Impacts
• Examples of evaluation components
• Inputs tracked numbers of attendees at events
• Processes observed community events materials
  distributed, lectures given, discussions held
• Outputs assessed knowledge about cancer
• Outcomes tracked cancer screenings, tracked
  decreased disease stigmatization
• Impacts tracked cancer survivors serving as
  volunteers for the program

11
Selected Evaluation Results
Estimated duplicate counts 40 Ministers
Includes breast, uterus, liver, prostate and
colon cancer screenings
12
5. Conclusions

• The program and the evaluation blended
  theoretical components from evaluation science
  and from the area of health education/disease
  prevention.
• The program set measurable objectives. These
  anchors provided a basis to judge achievements.
• The evaluation involved empirical assessments of
  inputs, processes, outputs, outcomes and impacts.
• The program planners/implementers from the
  Chinese and Korean communities were partners with
  the evaluation team, from start to finish.
• The programs continue today, with a smaller
  funding base

13
6. Audience Inputs and Comments

• Questions about the program
• Questions about the evaluation
• Questions about the results
• General questions about evaluation
• Any other general questions for discussion

14
Evaluation Next Steps

• New book available soon
• ?????? ???????? ??????????? ? ????????
• ????????? ?.???????, ?.?????????, ?.????????