Communicating Evaluation Results: Working Toward Sustainability - PowerPoint PPT Presentation


PPT – Communicating Evaluation Results: Working Toward Sustainability PowerPoint presentation | free to view - id: 180c5c-ZDc1Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Communicating Evaluation Results: Working Toward Sustainability


Communicating Evaluation Results: Working Toward Sustainability. Michele Cummins. SSRE ... Effective Communication Strategies for Key Sustainability Decision ... – PowerPoint PPT presentation

Number of Views:49
Avg rating:3.0/5.0
Slides: 22
Provided by: yonca2


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Communicating Evaluation Results: Working Toward Sustainability

Communicating Evaluation Results Working Toward
  • Michele Cummins
  • SSRE
  • Kim Netter

  • Introduction
  • Sustainability and Evaluation
  • Evaluation Data for Sustainability
  • Communications Plan
  • Incorporating Evaluation Data into Communications
  • Effective Communication Strategies for Key
    Sustainability Decision-makers

  • The capacity of a project to continue to deliver
    its intended benefits over a long period of time
    (World Banks definition in Bamberger and Cheema,
  • The ability of a project to deliver an
    appropriate level of benefits for an extended
    period of time after major financial, managerial,
    and technical assistance from an external donor
    is terminated (US Agency for International
    Development, 1988).

  • Systematic collection of information about
    program activities, characteristics, and outcomes
    for use to reduce uncertainty, improve
    effectiveness, and make decisions.

Evaluation Based on What?
  • You can never really say what youve
    accomplished, or whether youve accomplished
    anything at all, unless you have very specific
    quantifiable goals against which you can measure
    your effect.
  • Carl Safina,
  • National Audubon Society

What Can Evaluations Tell Us?
  • Whether we are doing what we intended to do
    and/or serving those who we intended to serve
  • Whether our services are appropriate
  • Whether our clients are satisfied with our
  • Whether our services are achieving intended, and
    perhaps unintended, results
  • Whether we are worth sustaining

Four Types of Evaluation Data
Process Outcome Impact Cost Effectiveness/Benefit
Process, Outcome, Impact, Cost
Effectiveness/Benefit Data
  • Process
  • Implementation (e.g., amount, nature,
  • Outcome
  • Short-term effects (e.g., changes in knowledge,
    attitudes, skills)
  • Focus on quality and effectiveness of activities
  • Impact
  • Long-term effects (e.g., changes in behavior,
    system change)
  • Focus on quality and effectiveness of strategies
  • Cost Effectiveness/Benefit
  • Relationship between program costs and program
    effectiveness or benefits

(No Transcript)
Characteristics of Evaluation Data
  • Quantitative numerical (e.g., test scores,
    attendance rates, survey scales)
  • Qualitative narrative (e.g., written
    descriptions, testimonials, success stories, case
    studies, focus groups, key informant interviews)
  • Comparative compared to others
  • Trend compared to the past
  • Experimental mixture of comparative and trend
    (e.g., pretest and posttest with a
    control/comparison group)
  • Sub-Group differences in effects (e.g., by
    gender, grade)
  • Proxy Surrogate/substitute (used when cost,
    complexity or timeliness prevent a result from
    being measured directly)

Process Data for Sustainability
  • Strategic Management
  • Implemented on time
  • Implemented as planned, or with changes that were
    based on sound reasoning
  • Fiscal Management
  • Implemented on budget
  • Delivery
  • Provided X number of services which was
    above/at/below the target level
  • Served X number of people, X percent of whom
    were from the target population

Outcome Data for Sustainability
  • Client Ratings
  • Client assessments of satisfaction, utility,
    relevance, appropriateness, etc.
  • Opportune use of qualitative information
  • Immediate Client Changes
  • Improvements in service-related knowledge,
    attitudes, skills, intended or current behavior

Impact Data for Sustainability
  • Long-Term Client Changes
  • Impact on service-related behavior
  • Strongest evidence
  • Not always available
  • System Changes
  • Evidence of enduring, system-level changes (e.g.,
    policy implementation or change)
  • Often qualitative
  • Not always available

Cost Effectiveness/Benefit Data for Sustainability
  • Cost Effectiveness
  • Showing that, dollar for dollar, certain
    programs, techniques, or strategies are as or
    more effective than others
  • Cost Benefit
  • Comparing the level of cost and associated
    benefit between programs
  • Discovering whether program costs are less than,
    equal to, or greater than associated benefits
    represented monetarily

Communications Plan
  • What are you trying to get done?
  • Who can get it done for you?
  • What do you have to convey?
  • Who should carry the message?

Message Triangle
Incorporating Evaluation Data into Message
  • Set the Stage explain the problem
  • demographics
  • data
  • paint the picture
  • Describe the Solution and Results explain what
    you do and what youve accomplished
  • program components
  • program goals
  • process, outcome, impact, and/or cost
    effectiveness/benefit data
  • combine quantitative and qualitative when
  • Detail What You Need call to action
  • funding
  • champions
  • policy change

Communicating Evaluation Results Effectively
  • Be honest and accurate
  • Make it real
  • Make it broad
  • Make it clear and concise
  • Layer information allow the audience to go as
    shallow or deep as they want
  • Choose the appropriate method for your audience

Choosing Appropriate Methods
Adapted from (1) Borden, DeBord, and Snipes and
(2) Morris, Gibson, and Freeman.
Preferences of Policy Makers Who Make
Sustainability Decisions
  • Want
  • Material relevant to current issues
  • Material related to real people
  • Short bulleted paragraphs instead of large text
  • Charts or graphs to illustrate key points
  • Brief reports/summaries or fact sheets, with more
    substantive back-up information available
  • Dont Want
  • Material that is too long, dense, or detailed
  • Material that is too theoretical, technical, or
  • Material that is biased or not objective
  • Audiotapes, audioconferences, e-mails, press

Adapted from Sorian and Baugh.
  • Bamberger, M. and Cheema, S. Case studies of
    project sustainability Implications for policy
    and operations from Asian experience.
    Washington, DC The World Bank, 1990.
  • Borden, L., DeBord, K., and Snipes, S. Beyond
    data. Department of Family and Consumer Sciences
    at North Carolina State University. Available
    online at http//
    data/ prfig2.htm. Accessed April 23, 2004.
  • Morris, L.L., Gibson, C.T., and Freeman, M.E.
    How to communicate evaluation findings. Newberry
    Park, CA Sage, 1987.
  • Sorian, R. and Baugh, T. Power of information
    closing the gap between research and policy.
    Health affairs. 21(2) 264-273, 2002.
  • US Agency for International Development.
    Sustainability of development programs A
    compendium of donor experience. Washington, DC
    USAID, 1988.