Building and Implementing Government Capacity to Deliver Social Good The Seven Deadly Sins as Seen f

1
Building and Implementing Government Capacity to
Deliver Social Good ---The Seven Deadly Sins as
Seen from an Evaluation Perspective - and How to
Avoid Them

• Dr. David Hunter, Hunter Consulting
• Dr. Steffen Bohni Nielsen, Rambøll Management

2
The Seven Sins

• First Sin Implementing Government Programs with
  No Evidence that they Will or Can Work
• Second Sin Implementing Programs with
  Nonexistent or Weak Theories of Change (that must
  address organizational capacities and resources,
  as well as program features)
• Third Sin Reducing Evaluation Systems to
  Accountability Systems
• Fourth Sin Drive-By Evaluations in Service of
  Accountability
• Fifth Sin Premature External Evaluations
  (without having built internal capacity)
• Sixth Sin Building Top-Down, Control-Oriented
  Performance Management Systems
• Seventh Sin Failing to Conduct Rigorous External
  Evaluations When They are Necessary and
  Appropriate

3
First Sin Implementing Government Programs with
No Evidence that they Will or Can Work

• The case
• US Dept. of Education Moving with Math,
  MATHThematics, the Algebra Projects
• How to avoid sinning
• Manage and use evaluative knowledge of prior
  interventions to design new interventions

4
Second Sin Implementing Programs with
Nonexistent or Weak Theories of Change

• The case
• D.A.R.E.
• How to avoid sinning
• Construct a plausible theory of change in the
  design phase
• Utilize existing evaluation findings to assess
  its plausibility

5
Third Sin Reducing Evaluation Systems to
Accountability Systems

• The case
• US Dept. of Education 21st Century Community
  Learning Centers
• How to avoid sinning
• Carefully frame the use of evaluation findings

6
Fourth Sin Drive-By Evaluations in Service of
Accountability

• The case
• Drive-by Evaluations build no capacity to
  incorporate evaluation-based practices in
  performance managment locally (external
  evaluators build their own capacity to collect
  local data) and simply deliver verdicts
  regarding program results to funders.
• Too narrow focus on outcomes (vs. Learning about
  what makes programs work or fail)
• Example No Child Left Behind (N.C.L.B.)
• How to avoid sinning
• Evaluation Capacity Building (ECB). Support
  local development and use of evaluative knowledge
  in service of performance management. Apply
  impact evaluations models at appropriate times
  and purposefully, not as gold standard per se.

7
Fifth Sin Premature External Evaluations

• The case
• Community Partnership for Protecting Children
  (C.P.P.C.)
• How to avoid sinning
• (More) Evaluation Capacity Building (ECB)
• Diligent performance management
• Indicators of readiness for external evaluation
  (a) program delivery meets performance standards
  across all sites, (b) programming is delivered
  reliably, and (c) delivery is sustained (for at
  least a few years)

8
Sixth Sin Building Top-Down, Control-Oriented
Performance Management Systems

• The case
• Mødrehjælpen (Mothers Aid)
• How to avoid sinning
• ToC as part of organisational development
• Frontline staff buy-in
• Data driven performance management at all
  organizational levels
• Organizational learning

9
Seventh Sin Failing to Conduct Rigorous External
Evaluations When They are Necessary and
Appropriate

• The case
• US G.P.R.A.
• DK Forthcoming Ministry of Finance guidelines
• How to avoid sinning
• Legislation or codified procedures designating
  the tactical and strategic use of evaluation.