Evidence Based Chronic Disease Prevention Module 8: Evaluating The Program Or Policy

1
Evidence Based ChronicDisease Prevention Module
8 EvaluatingThe Program Or Policy

• Presented by Karen Peters, DrPH

2
Objectives

• Understand the basic components of program
  evaluation.
• Describe the differences and unique contributions
  of quantitative and qualitative evaluation.
• Understand the concepts of measurement validity
  and reliability.

3
Develop statement of the issue
Determine what is known in the literature
Dissemination
Quantify the issue
Evaluate the program or policy
Develop program or policy options
Develop an action plan
4
Overview

• The whats and whys of evaluation
• Study designs and measurement issues
• Types of evaluation
• Quantitative
• Qualitative
• Organizational issues in evaluation
• Infeasible to provide in-depth discussion
  mechanics
• e.g., sampling/instrument development
• Can show basic components in an evaluation, what
  to look for, where to turn for help

5
What is Evaluation?

• A process that attempts to determine as
  systematically and objectively as possible the
  relevance, effectiveness, and impact of
  activities in light of their objectives.
• From Last JM. A Dictionary of Epidemiology.
  Third Edition. New York Oxford Press 1995.
• Complex and diverse field

6
Some questions to consider?

• What are the most significant challenges you face
  in program evaluation?
• In program planning when should you begin
  planning an evaluation?

7
Why Evaluate?

• Public health agencies need to be accountable
• Basis for choices when resources are limited
• Helps to determine costs and benefits
• Source of information for improving programs and
  policies
• Increasingly mandated by funders/policy makers
• Convincing funders is not always easy
• Leads to research questions that can be tested in
  other studies/programs

8
New Directions

• Social programs have become more broad, complex
  and interactive
• Seek to bring about changes in community
  capacity, social support, decision-making,
  control over resources and individual behavior
• Time to supplement traditional strategies with
  new approaches reflecting complexity of
  community-based initiatives

9
New Directions

• More flexible evaluation approaches can play role
  in building community capacity and self
  determination
• Need to re-direct program evaluation toward
  community-based public health values
• Traditional evaluation conducted by experts to
  determine if program objectives met,
  strengths/weaknesses/replicability and
  contribution to scientific knowledge

10
New Directions

• Some evaluators believe communities lack skills
  to design, engage in and interpret evaluations
• However, experts may lack insight/ flexibility
  needed to capture essence of community projects
  or to answer questions raised by communities and
  CBOs

11
New Directions

• Community-based evaluation perspective involves
  more participatory and inclusive process that
  incorporates the values, knowledge, expertise and
  interests of the community and uses evaluation as
  a tool for community capacity building
• Community involved as full/equal partner allows
  for development of more relevant program
  success measures and produces data that are
  useful in community settings

12
New Directions

• Evaluation is 1 part of broader planning process
  but can help in
• reflecting on progress
• document where going, where coming from
• share what worked and what did not with other
  communities
• demonstrate need for targeted resources to
  address community issues
• illustrate impact of community based initiatives
  to decision/policy makers
• provide information on developing meaningful
  community based indicators

13
Research Phases Model(Greenwald and Cullen, 1985)

• Phase 1 Hypothesis Development
• Ex Link between sedentary behavior - obesity
• Phase 2 Methods Development
• Pilot test of intervention to increase physical
  activity and validation of measures
• Phase 3 Controlled Intx Trial (Efficacy)
• Small scale randomized trial of physical activity
• Phase 4 Defined Population (Effectiveness)
• Larger scale trial of PA in populations of
  interest
• Phase 5 Demonstration (Dissemination)
• Evaluation of results of PA program

14
Evaluation Models

• For Practitioners
• Models identify key factors to consider when
  developing/selecting health behavior programs
• Factors to focus on when reading the literature
• For Researchers
• Models identify important dimensions to be
  included in program evaluations

15
Evaluation Models

• 2 Major Frameworks (there are many others)
• For Process and Formative Evaluation
• CDC Framework for Public Health Programs
• For Impact and Outcome Evaluation
• RE-AIM Framework

16
Framework for Program Evaluation

• 2 year process by CDC
• designed as a framework for ongoing, practical
  program evaluation
• can be integrated with routine program program
  operations
• involves input from program staff, community
  members, other stakeholders, not just evaluation
  experts
• Involves 6 basic steps and 4 broad evaluation
  standards

17
Broad Evaluation Framework (ala CDC)

• Engage stakeholders
• Describe the program
• Focus the evaluation design
• Gather and analyze evidence
• Justify conclusions
• Ensure use and share lessons learned
• CDC Framework for Program Evaluation in Public
  Health, 9/17/99 and Center for Advancement of
  Community Based Public Health, June 2000

18
CDC Framework for Evaluation

• 4 Evaluation standards
• Guidelines that can help assess whether an
  evaluation is well designed
• Utility Is the evaluation useful?
• Does the evaluation answer questions that are
  relevant to the stakeholders
• Feasibility Is the evaluation practical?
• Is the evaluation realistic and cost-effective

19
CDC Framework for Evaluation

• Propriety Is the evaluation ethical?
• Does the evaluation consider the rights and
  interests of those involved and affected
• Accuracy Is the evaluation correct?
• Do the evaluation findings convey information
  that is correct and technically adequate

20
CDC Framework for Evaluation

• Step 1 Engage the stakeholders
• Stakeholders - those involved in implementing the
  program and those served or affected by it
  including decision-makers who can do something
  with the results
• Standards for step 1 Utility and Propriety
• Utility Have you identified individuals and
  organizations affected by the evaluation?
• Are those involved in the evaluation trustworthy
  and competent?

21
CDC Framework for Evaluation

• Propriety
• Is there an explicit, written agreement about
  what is to be done, how, by whom and when?
• Does the evaluation design protect the rights and
  welfare of those involved?
• Are the individuals who are conducting the
  evaluation interacting respectfully with
  stakeholders?
• Have you discussed conflicts of interest openly
  and honestly?

22
CDC Evaluation Framework

• Step 2 Describe the program
• summarize the program being evaluated with a
  statement of need which includes expectations, a
  logic model, specifying resources available to
  conduct program activities and how program fits
  into larger organizational/community context
• A good description allows the program to be
  compared to similar efforts and makes it easier
  to figure out what parts brought about what
  effects

23
CDC Evaluation Framework

• Standards for Step 2 Accuracy/Propriety
• Accuracy Have you clearly and accurately
  described the program
• Have you documented the program context
• Propriety Is the evaluation complete and fair
• Does it assess program strengths and weaknesses

24
CDC Evaluation Framework

• Step 3 Focus the evaluation design
• Specify the evaluations overall intent
• Determine who and what the evaluation is for
  (users and uses)
• What questions the evaluation should examine
• What methods are best to answer the questions

25
CDC Evaluation Framework

• Purpose of evaluation depends on programs stage
  of development
• New or developing program - feasibility of
  intervention approach
• Implementation - fine tuning or changes needed
• Established program - assess program effects
• Standards for Step 3
• Feasibility
• Propriety
• Accuracy

26
CDC Evaluation Framework

• Feasibility
• Have you considered the political interests/needs
  of groups and obtained buy-in
• Does the information produced justify the costs
• Are evaluation procedures practical
• Propriety
• Does evaluation design help to identify needs
• Are costs guided by sound/ethical accountability
  procedures
• Accuracy
• Is there an accurate description of evaluation
  purposes and procedures

27
CDC Evaluation Framework

• Step 4 Gather credible evidence
• Need well rounded picture of the program
• Develop indicators that translate program
  concepts into specific measures
• Use multiple sources of evidence that reflect
  different perspectives about the program
• Techniques used to gather and handle evidence
  should be compatible with cultural conditions in
  each program setting

28
CDC Evaluation Framework

• Standards in Step 4 Utility and Accuracy
• Utility Are you collecting information that
  addresses pertinent program issues and is
  responsive to stakeholder needs
• Accuracy Have you adequately described your
  sources of information
• Do data collection procedures address internal
  validity and reliability issues
• Is there a system in place for identifying and
  correcting errors

29
CDC Evaluation Framework

• Step 5 Justify Conclusions
• Involves making claims about a program based on
  evidence gathered
• Stakeholder values provide basis for making
  judgements about program merits/performance
• Conclusions are based on analysis,synthesis and
  interpretation of information to detect patterns
  and result in recommendations
• Reaching good conclusions requires variety of
  stakeholder perspectives

30
CDC Evaluation Framework

• Standards for Step 5 Accuracy/Utility
• Accuracy Has data analysis process been
  effective in answering key evaluation questions
• Can you explicitly justify your conclusions
• Utility Have you carefully described the
  perspectives, procedures and rationale used to
  interpret the findings

31
CDC Evaluation Framework

• Step 6 Ensure use, share lessons learned
• Make sure stakeholders understand the evaluation
  procedure and findings
• All participants should have the opportunity to
  provide feedback
• Evaluators should provide any needed follow-up
• Use a variety of communication strategies to
  disseminate evaluation results in a timely and
  unbiased manner

32
CDC Evaluation Framework

• Standards for Step 6 Utility, Propriety,
  Accuracy
• Utility - do evaluation reports describe the
  program context, purpose, procedures and findings
  clearly?
• Propriety - Have evaluators made sure findings
  (including limitations) are accessible to
  everyone affected by the program
• Accuracy - do evaluation reports reflect the
  findings fairly and impartially?

33
RE-AIM Evaluation FrameworkGlasgow, Vogt, Boles,
1999Glasgow, McKay, Piette, Reynolds, 2001

• Reach, Efficacy or Effectiveness (depending on
  research phase), Adoption, Implementation,
  Maintenance
• RE-AIM relies on 2 comprehensive models
• PRECEDE-PROCEED (Green Kreuter 1999)
• Diffusion Theory (Rogers 1995 Nutbeam 1996)

34
Dimensions of RE-AIM

• Reach
• Individual Level
• What of potentially eligible participants will
  take part
• How representative are they?
• Efficacy or Effectiveness
• Individual Level
• What was the impact on all who began?
• What was the impact on intermediate primary
  outcomes?
• What was the positive/negative (unintended)
  outcomes, including quality of life?

35
Dimensions of RE-AIM

• Adoption
• Setting Level
• What of settings/intervention agents will
  participate? (worksites, schools, educators,
  nurses)
• How representative are they?
• Implementation
• Setting or Agent Level
• To what extent were the intervention components
  delivered as intended (in the protocol), when
  conducted in applied settings by non researchers

36
Dimensions of RE-AIM

• Maintenance
• Individual and Setting Levels
• Individual Level What are the long term effects
  (minimum 6-12 months following intervention)?
• Setting Level To what extent are different
  intervention components continued or
  institutionalized?

37
Common Challenges and Suggested Strategies Using
RE-AIM Framework

• Reach
• Challenge Not including a relevant, high risk,
  or representative sample
• Strategy Use population-based recruitment or
  over sample high risk groups, reduce exclusion
  criteria
• Efficacy or Effectiveness
• Challenge Ambiguous outcomes
• Strategy Assess broader set of outcomes, conduct
  subgroup analyses, use different assessment points

38
Common Challenges and Suggested Strategies Using
RE-AIM Framework

• Adoption
• Challenge Program never adopted or endorsed -
  used only in academic settings
• Strategy Involve participants in all phases,
  approach numerous settings early on while
  revision is still possible
• Implementation
• Challenge Protocols not delivered as intended
  (Type III error)
• Strategy Assess treatment as too complicated,
  intensive, incompatible involve non-researchers

39
Common Challenges and Suggested Strategies Using
RE-AIM Framework

• Maintenance
• Challenge Program or effects not maintained over
  time
• Strategy Include maintenance phase in protocol
  and evaluation plans Leave treatment behind
  after study and plan for institutionalization

40
Common Study/Evaluation Designs

• Experimental/randomized
• Quasi-experimental
• Time-series
• Use of existing data

41
Study/Evaluation Designs

• Quasi-experimental
• Increasing attention
• At least one intervention and one comparison
  group, without randomization
• Appeal of intervening thru intact social groups
• See Koepsell chapter in readings (in Module 4)

42
Study/Evaluation Designs

• Use the best designs feasible
• Pre- post-data
• Comparison groups
• Complete program records
• Reliable and valid measures
• Proper analytic techniques
• Review principles/tools from Goodman, page 39 (in
  Module 8)

43
Challenges in CommunityWide Studies

• Varying degrees of intervention exposure
• Running programs in multiple locations
• Accounting for community-level variance
• Lack of sensitivity of the community
• Concepts of participatory research
• Equity, collective decision making
• High-quality, ethical research
• Addressing social inequalities
• Maximize learning opportunities
• See Goodman article on community capacity

44
Challenges in CommunityWide Studies

• Community-level variance
• Individuals in communities, neighborhoods,
  schools, worksites are correlated
• ICC (intra-class correlation coefficient)
• People have related characteristics (not
  independent)
• In practical terms, means increased chance of
  Type I error (saying there is a difference when
  there really is not)

45
Measurement Issues

• Components of a good evaluation
• Adequate sample size
• High validity
• High reliability
• Sample size considerations
• Number of communities
• Number of individuals per community
• Increasing number of communities versus number of
  individuals per community
• Can rely on simple, accessible programs like Epi
  Info

46
Concepts of Validity and Reliability and their
Importance

• Measurement Issues
• Evaluation threats
• Validity
• Is the instrument or design measuring exactly
  what was intended? (Self report vs.... biologic
  test)
• Reliability
• Is the measurement being conducted consistently?
  (Face to face vs...... telephone, different
  interviewers)

47
Measurement Issues

• Validity best available approximation to the
  truth
• Internal Validity
• The extent of causality (the effects are really
  attributable to the program)
• External Validity
• The extent of generalizability
• Importance?? (so what???)

48
Measurement Issues

• Major threats to validity
• Low statistical power
• Violated assumptions in statistical tests
• Reliability of measures
• Reliability of treatment implementation
• Random confounders in the experiment
• Random heterogeneity of respondents
• adapted from Cook and Campbell, 1979

49
Measurement Issues

• Reliability (repeatability)
• Consistency in measurement
• Multiple types
• Inter-observer
• Test-retest
• Internal consistency

50
Measurement Issues Examples

• Validity
• Self-reported rate of having a health
  professional check for hemoglobin A1C among
  diabetics in an intervention program compared
  with clinic records
• Reliability
• Test-retest data from the BRFSS on self-report of
  seeing a health care professional in the last
  year for diabetes among diabetics in Illinois

51
Measurement Issues Ensuring Validity and
Reliability in Evaluation

• Literature/contacting researchers may show you
  accepted methods
• Multiple statistical methods available to report
  validity and reliability
• Evaluation instruments often need community
  contouring
• Participatory methods may prevent use of existing
  instruments/questions

52
Four Purposes of Evaluation

• Needs Assessment
• Process
• Impact
• Outcome

53
How do you decide?

• Use data available
• What does the data tell you?
• What other information do you want?
• What other programs are already available in
  different locations?
• Reality constraints Staffing, resources, time

54
Needs Assessment

• Diagnostic Evaluation
• Context Evaluation
• feedback on knowledge, attitudes, risk behaviors,
  health status, and perceived needs of the target
  population and of the status of available health
  promotion programs

55
Process Evaluation

• Formative evaluation
• feedback on program implementation site
  response participant response including
  appropriateness of materials, methods,content
  practitioner response, and personnel competency

56
Example IL Arthritis Awareness

• 3 components
• brochure alone
• brochure with community outreach
• brochure, community outreach, physician education
• What would you want to know about each component?

57
Types of Evaluation Process evaluation

• Field of Dreams evaluation
• If you build it, will they come?
• Shorter-term feedback on program implementation,
  content, methods, participant response,
  practitioner response
• What is working, what is not working
• Uses quantitative or qualitative data
• Data usually involves counts, not rates or ratios

58
Types of Evaluation

• Considerations for process evaluation
• Sources of data
• Program data
• Limitations of data
• Time frame
• Availability
• Example
• Number of diabetics being obtaining foot
  examinations statewide through primary care
  providers

59
Impact Evaluation

• Summative evaluation
• long or short term feedback on knowledge,
  attitudes, beliefs and behavior change (KABBBs)
  of participants (skills development), programs
  and policies of organizations and governments
• Time 1 - decide what to include in intervention
• Time 2 - did the intervention make a difference
• Conduct survey or interview participants

60
Impact Evaluation

• Uses quantitative or qualitative data
• Probably more realistic endpoints for most public
  health programs and policies

61
Impact Evaluation

• Considerations for impact evaluation
• Sources of data
• Surveillance or program data
• Limitations of data (validity and reliability)
• Time frame
• Availability
• Example
• Eye examination rates of diabetics in Illinois

62
Types of Evaluation

• Outcome evaluation
• Long-term feedback on health status, morbidity,
  mortality, disability or quality of life
• Uses quantitative data
• Present throughout integrated strategic plan
• Also called summative evaluation

63
Types of Evaluation

• Considerations for outcome evaluation
• Sources of data
• Routine surveillance data
• Limitations of data (validity and reliability)
• Time frame
• Availability
• IDPH IPLAN website
• Example
• Rates of end stage renal disease among diabetics
  in Illinois

64
Types of Evaluation Outcome evaluation

• Some move toward standardizing (outcome)
  indicatorse.g., CDCs consensus indicators
• What if yours is not on the list?
• Race-ethnicity-specific infant mortality rate
• Motor vehicle crash death rate
• Work-related injury death rate
• Suicide rate
• Lung cancer death rate
• Breast cancer death rate
• Cardiovascular disease death rate
• Homicide rate

65
Types of Evaluation Outcome evaluation

• All-cause mortality rate
• AIDS incidence
• Measles incidence
• Tuberculosis incidence
• Syphilis incidence
• Incidence of low birth weight
• Births to adolescents
• Prenatal care
• Childhood poverty
• Proportion of persons living in counties
  exceeding EPA standards for air quality

66
Types of Evaluation Outcome/impact evaluation

• Common public health data sources
• Vital events registries
• Disease specific registries
• Population and house census
• Routine health services records
• Health programs delivery records
• Hospital discharge data
• Disease notification within surveillance systems
• Sample surveys within surveillance systems
• Other sample surveys
• Other data banks from programs outside the health
  sector

67
Data Questions for Consideration

• What type of evaluation is this?
• How would you analyze these data?
• What would be your initial conclusions?
• What other data might support or detract from
  your conclusions?
• What are some limitations of your risk factor
  data?

68
Types of Evaluation

• Outcome
• Doing the Right
  Things
• Impact
• (performance measures)
• Process Doing Things Right

69
Evaluation Framework

• Program
• -
  Instructors?
• - Content?
• Process -
  Methods?
• - Time
  allotment?
• -
  Materials?
• Behavior
• - Knowledge gain?
• Evaluation Impact -Attitude change?
• Types -Habit change?
• -Skill development?
• Health
• - Mortality?
• Outcome -Morbidity?
• -Disability?
• (Adapted from Green et al, 1980) -Quality of
  life

70
Evaluation Polarities

• Formative Summative
• Qualitative Quantitative
• Process Outcome
• Impact

71
Types of Evaluation

• Quantitative versus qualitative methods
• Avoid choosing one or the other
• Generally more familiarity with quantitative
  methods

72
Quantitative vs.... Qualitative

• Quantitative (numbers) surveillance data,
  surveys, records
• Qualitative (words) observations, in-depth
  individual interviews, group interviews, focus
  groups, diaries

73
Why Qualitative?

• When there are no quantitative instruments
  available
• When you are not sure if the measures are
  appropriate for the population you are working
  with
• When you are not sure if you are asking the right
  questions
• When you want to understand program processes not
  just impact and outcomes

74
Why Qualitative?

• When you want more in depth information about
  program implementation -
• What do clients experience,
• what services are provided to clients,
• How is the program organized?
• What do staff do? How has the program developed?
• When you want detailed, descriptive information
  about the program for the purpose of improving
  the program

75
Observation Dimensions

• Role of observer
• full participant, partial participant,
  onlooker/observe as outsider
• Covert or overt
• Do others know you are observing?
• Duration of observation
• Short term vs..... long term
• Focus of observation
• narrow, single component of program vs..... whole

76
Types of Questions You Can Ask

• Behavioral/experience
• Opinion/value
• Feeling
• Knowledge
• Sensory
• Demographic/background
• (ALL from past, present, future)

77
In Depth Interviewing Types

• Informal conversational interview
• General interview guide
• Standardized open-ended interview

78
Informal Conversational Interview

• No interview guide no predetermined set of
  questions
• Particularly useful when do not know what types
  of questions to ask and when yo will be in the
  environment for some time
• Each interview builds on the previous one -
  flexibility and responsiveness to previous
  experiences and the environment

79
Informal Conversational Interview

• Requires a long time to get systematic
  information
• Open to interviewer effects
• tired
• conversational skills
• capacity to build rapport

80
Interview Guide

• List of questions or issues to be explored - a
  framework
• Can get similar information from several
  individuals
• General topics and probes
• General outline to discuss on predetermined
  subject
• Maintain conversational quality

81
Standard Open-Ended Interview

• Set of carefully worded and arranged questions
• Less flexibility of issues to explore or probing
• Minimizes variation in questions asked, thus
  reduces bias
• Useful when have several interviewers

82
Focus Group and Group Interviews

• Relatively homogeneous group
• 6-10 people
• Semi-structured
• Focus group - build on each others ideas -
  advantage of group processcan see influence of
  social networks on issue at hand
• Group interview - several people together, not
  necessarily take advantage of group process

83
Wording of Questions

• Open ended - do not presuppose dimensions of
  feeling, thoughts, or experiences
• Clear - use appropriate language, reflect back
  language you hear
• Singular questions - can preface with going to
  ask about strengths, weaknesses, likes, dislikes
  - then ask each individually
• Neutrality re subject and rapport with individual

84
Wording of Questions

• Acknowledge multiple experiences - can provide
  illustrations or examples some have said this,
  others that, still others have different opinions
  - what do you think?
• Avoid leading questions

85
Probes

• Can you give me an example of that?
• Can I stop you here for a minutecan you tell me
  a bit more about x can re-direct conversation
• Does anyone else have other experiences with this?

86
Recording Data

• Permission and consent
• Notes - during and after the interview include
  your feelings tired/excited how did the
  interview seem to you?
• Note surroundings
• Tape recording - make sure it is on, make sure it
  works

87
Sampling

• Purposeful sampling - notion of gaining
  information from those who have it
• extreme or deviant cases - outstanding successes
  or notable failures
• maximum variation - look for common themes across
  multiple participants or programs - small sample,
  great diversity
• homogenous samples - individual or focus groups

88
Sampling

• Snowball or chain sampling
• recommended informants - who else should I speak
  with
• Criterion sampling
• all cases who meet some predetermined criteria

89
Analysis

• From transcription and notes
• Focused coding - with predetermined categories in
  mind
• Open coding - categories and themes from the data
  itself
• Multiple coders
• Label so you can go back to context
• Triangulation

90
Reliability and Validity

• Neutrality rather than objective and subjective
• impartial, not predisposed toward certain
  findings ahead of time, no predetermined results
  to support
• Notion of generalizability vs.... extrapolation
• generalizable - impossible because nothing is
  context free
• extrapolation - speculations on the likely
  applicability of findings to other situations
  under similar, not identical, conditions

91
Other Considerations in Qualitative Evaluation

• Involve stakeholders in development of program
  objectives and evaluation questions
• Evaluation requires clear program objectives
• Measure program processes, impacts, and outcomes
  using measures appropriate for the questions
  asked
• Expect frustration from those collecting data,
  resistance from those feeling judged

92
Organizational Issues/Summary

• Program personnel may be threatened by the
  evaluation
• Need to maintain objectivity
• Involvement may reduce resistance but may
  threaten objectivity
• Trade-off of comprehensive evaluation versus
  nothing at all
• Think of 10 Rule as you design and implement
  programs

93
Organizational Issues/Summary

• How can I do evaluation when theres so much
  real work to do?
• Independent (outside) evaluation may be useful
• What to look for in a good evaluation
• Remember the use of multi-disciplinary teams

94
A Word on Coalitions and Partnerships

• Seems to be an interest in and emphasis on the
  use and effectiveness of coalitions as part of
  public health programming
• Little is known currently about impact and
  outcome measures for coalitions
• More is known about process of coalitions
• Both qualitative and quantitative measures are
  needed in this area

95
Other Resources on Evaluation

• World Wide Web Virtual Library Evaluation
• http//user.berlin.de/alfio.cermi/index.html
• CDC Evaluation Resources
• http//www.cdc.gov/eval/over.htm
• CDC Framework for Program Evaluation in Public
  Health
• http//www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1/
  htm
• Center for the Adv. of Comm Based Public Health
  (www.cbph.org)
• University of Toronto (http//www.utoronto.ca/shp/
  hcu)
• Univ.of WI Ext http//www.uwex.edu/ces/pdante/eval
  uat/htm
• Kellogg Fndt http//www.wkkf.org/Publications/eva
  lhdbk/

96
Course Objectives

• Construct a concise, measurable statement of a
  public health issue or statement
• Determine what is known in the scientific
  literature
• Describe public health issues using quantitative
  data sources
• Generate public health program or policy options
• Create public health program or policy action
  plans
• Construct a logic model
• Evaluate a public health program