Using a results-based evaluation methodology in a public sector environment that focuses on activities - PowerPoint PPT Presentation

Loading...

PPT – Using a results-based evaluation methodology in a public sector environment that focuses on activities PowerPoint presentation | free to download - id: 8095f8-MThkN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Using a results-based evaluation methodology in a public sector environment that focuses on activities

Description:

Using a results-based evaluation methodology in a public sector environment that focuses on activities – PowerPoint PPT presentation

Number of Views:4
Avg rating:3.0/5.0

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using a results-based evaluation methodology in a public sector environment that focuses on activities


1
Using a results-based evaluation methodology in a
public sector environment that focuses on
activities
2
Terms of Reference - Example
  • To conduct an evaluation of the effectiveness and
    impact of the DLGTA support over the past five
    years using a results based management (RBM)
    approach.
  • Investigation into the performance and management
    of the community libraries recapitalisation
    programme
  • The department of economic development agency
    micro-level monitoring and evaluation system.
  • Evaluation of the soul buddyz clubs

3
What are we faced with at the commencement of the
monitoring and evaluation assignment
4
Planning StatementsStrategic Goals - Examples
  • To invest and ensure the provision of quality
    social welfare services to children including
    those in need of care and protection.
  • Facilitate integrated and responsive governance
    in a developmental state.
  • Promote health, prevent and manage illness or
    conditions with emphasis on poverty, lifestyle,
    trauma and violence and psychosocial factors

5
Planning StatementsStrategic Objectives -
Examples
Strategic Objective Indicator
Ensure that municipalities meet basic needs of communities.
Lateral contribution Executive development learning framework. Capacity for better implementation of donor funded projects
Transformation Design and deliver innovative multimodal programmes.
6
Planning Statements Measurable Objectives -
Examples
  • Reduce infant and under five morbidity and
    mortality
  • To ensure effective leadership, management and
    administrative support to the department through
    the continuous refinement of organisational
    strategy and structure, in compliance with
    appropriate legislation and best practice.
  • Conduct campaigns on key nutrition priorities
    such as obesity
  • To promote growth, social development and poverty
    reduction through sound fiscal and financial
    policies, and the effective, efficient and
    appropriate allocation of public funds.

7
Results Statements Outcomes - Examples
Outcome Indicators
Enhance service delivery   Regulatory and support mechanisms for Municipal Councils and ward committees reformed and implemented
Maturity in ME system for cooperative governance A coordinated and functional M,RE system for Provincial and Local Government
Coordinated ME system for cooperative governance Regulatory framework for ME implemented
8
Results Statements Outputs - Example
Output indicators
Improved quality and quantity of measurable objectives and trend based performance indicators by departments and public entities Publication and implementation of a standard operating procedures manual
Local government budget Framework Integrity of the framework fiscal sustainability, structure and trends in fiscal indicators
Infrastructure delivery improvement programme Technical Assistants deployed in targeted provincial Departments
Enhance coordination across government for effective implementation of MFMA An agenda that ensures alignment of activities of national and government departments consistent with agreed Priorities
9
Where do we start
10
(No Transcript)
11
Where do we start
  • Review all planning documents these include
  • Strategic plans
  • Annual performance plans
  • Business Plans
  • Operational Plans
  • Map the planning statements, the indicators,
    targets, activities etc.
  • Identify all available data
  • Quarterly reports
  • Annual Reports
  • Baseline data
  • Reports developed for special studies conducted

12
Evaluation Parameters
Results Matrix
13
Reconstructing the Intervention Logic
  • Check whether the planning statements (goal,
    strategic objectives and measurable objectives)
    could be reformulated as results statements
    (output, outcome and impact), so they can be
    measured accordingly.
  • Check whether the sum of the planned
    components/support is sufficient to produce the
    intended result.
  • Explicitly describe the planning assumptions and
    minimize the risk of failure (did we assume too
    much?)
  • Map the indicators detailed in the Strategic
    Plans, APP and the Operational Plans to the
    remapped results matrix.
  • In trying to establish the model of evaluation
    and the methodology to be employed for the
    evaluation the following should be considered
  • remapping of the planning levels using the
    parameters governing logic models.
  • establish whether there are linkages between the
    activities, the outputs, the outcomes and the
    impacts.

14
Logic Model
ACTIVITIES
OUTPUTS
DIRECT OUTCOME
INTERMEDIATE OUTCOME
ACTIVITIES
OUTPUTS
ULTIMATE OUTCOME
ACTIVITIES
OUTPUTS
DIRECT OUTCOME
INPUTS
ACTIVITIES
OUTPUTS
INTERMEDIATE OUTCOME
INPUTS
ACTIVITIES
DIRECT OUTCOME
OUTPUTS
15
Would you tell me, please, which way I ought to
go from here? That depends a good deal on
where you want to get to, said the Cat. I dont
much care where said Alice. Then it doesnt
matter which way you go, said the Cat. so
long as I get SOMEWHERE, Alice added as an
explanation. Oh, youre sure to do that, said
the Cat, if you only walk long enough.
16
Evaluability Assessment
  • Evaluability assessment is a brief preliminary
    study to determine whether an evaluation would be
    useful and feasible.
  • Establish the discrepancies between the rhetoric
    and the reality according to Nay and Kay (1982)
    different levels of policy makers and programme
    managers may have different rhetorical models for
    the same programme and they cling to these models
    because it is perceived that approach to bring
    about change.
  • The planning framework utilised and the evidence
    of the logical linkages between the different
    levels of planning.
  • The appropriateness and plausibility of the
    Strategic goals and Strategic Objectives in terms
    of the legislative mandate and core functions of
    the province.

17
Evaluability Assessment
  • Assess the extent to which the intervention logic
    provides evaluators with operational benchmarks
    against which outputs, outcomes, impacts and
    assumptions can be evaluated.
  • Check whether necessary baseline and monitoring
    data is available.
  •  
  • An exploration of the evaluation models to
    determine the degree the models are relevant,
    appropriate and feasible in terms of
  • The availability of information routine, ad hoc
    and research information
  • The logic used to construct the programmes
  • The timing and relevance of such an evaluation.
  •  
  • Check the availability of key informants, such as
    planners, intervention staff, and target group
    representatives.
  • Assess the extent to which the evaluation
    questions can be answered, given the timing of
    the evaluation in relation to the current phase
    of the intervention cycle.

18
The Evaluation
19
Measuring change and inferring causality
  • Relevance
  • The extent to which a development intervention
    conforms to the needs and priorities of target
    groups and the policies of recipient countries
    and donors.
  • Sustainability
  • The continuation or longevity of benefits from a
    development intervention after the cessation of
    development assistance.
  • Efficiency
  • The extent to which the costs of a development
    intervention can be justified by its results,
    taking alternatives into account.
  • Assumes that there has been effectiveness
  • Other evaluation criteria
  • Appropriateness tailored to local needs and
    changing demands
  • Coverage evenness, comprehensiveness
  • Connectedness co-operation
  • Coherence and alignment

20
Components of an Evaluation
  • There are 3 general components to comprehensive
    evaluation
  • Process evaluation How was the strategy/
    programme implemented?
  • Outcome evaluation Did the strategy/ programme
    meet its objectives?
  • Impact evaluation Was the ultimate goal of the
    strategy/programme achieved?

21
  • Every program has

Every program evaluation should have
Impact Indicators
Goals Objectives Activities
Outcome Indicators
Process Indicators
22
  • Every Operational Plan/Programme has

Every evaluation should have
Impact Indicators
Goals Objectives Activities
Outcome Indicators
Process Indicators
23
What is Process Evaluation?
  • Process evaluation
  • Addresses how, and how well, the program is
    functioning
  • It can help to
  • Create a better learning environment
  • Improve presentation skills
  • Show accountability to funder
  • Reflect the target populations
  • Track service units

24
Process Evaluation cont
  • Key questions in process evaluation
  • Who is served?
  • What activities or services are provided?
  • Where, when, and how long is the program?
  • Identify how a product or outcome is produced
  • Identify strengths weaknesses of a program
  • Create detailed description of the program

25
  • Every Strategic Plan/Operational Plan/Programme
    has

Every program evaluation should have
Impact Indicators
Goals Objectives Activities
Outcome Indicators
Process Indicators
26
Outcome Evaluation
  • Outcome evaluation
  • Measures the extent to which a Strategy/
    programme produces its intended improvements
  • Examines effectiveness, goal attainment and
    unintended outcomes
  • In simple terms, Whats different as a result of
    your efforts?

27
Outcome Evaluation cont
  • Key questions in outcome evaluation
  • To what degree did the desired change(s) occur?
  • Outcomes can be immediate, intermediate or
    longer-term
  • Outcomes can be measured at the patient,
    provider, organization, or system level.

28
  • Every Strategic Plan/ Programme has

Every evaluation should have
Impact Indicators
Goals Objectives Activities
Outcome Indicators
Process Indicators
29
Impact Evaluation
  • Impact is sometimes used to mean ultimate
    outcome.
  • Impact is perhaps better defined as a longer-term
    or ultimate outcome.

30
Results-Based Evaluation
Results-Based Evaluation is an assessment of
a planned, ongoing, or completed intervention to
determine its relevance, efficiency,
effectiveness, impact and sustainability. The
intent is to incorporate lessons learned into the
decision-making process.
31
INFORMATION (MIS) USE
Focus should be knowledge generation
About PowerShow.com