Impact Pathways: An Approach for Understanding, Fostering and Evaluating Research-for-Development Outcomes

1
Impact Pathways An Approach for Understanding,
Fostering and Evaluating Research-for-Development
Outcomes

• Boru Douthwaite, Technology Policy Analyst
• Sophie Alvarez, Consultant
• Simon Cook, Leader BFPs
• Rick Davies, ME Specialist,
• Pamela George, CPWF Program Manager,
• John Howell, ME Specialist,
• Ronald Mackay, Professor Emeritus,
• Jorge Rubiano, National University of Colombia
• CIAT Seminar, 1st November 2006

2
Impact Pathways Matter
3
How change happens

• Improvements in poverty alleviation, food
  security and the state of natural resources
  result from dynamic, interactive, non-linear, and
  generally uncertain processes of innovation.
• EIARD, 2003
• EIARD represents a group of European donors
• 15 EU Countries plus Norway and Switzerland

4
Impact Pathways Approach

• People plan and implement projects on the basis
  of their change models - their implicit theories
  about how the world works
• If you can improve these theories you can improve
  the practice, making impact more likely
• Impact Pathways Approach A participatory
  approach for
• Making practitioners theories explicit about how
  they will achieve adoption and impact (impact
  pathways, program theory)
• Improving these theories
• Using those models / frameworks for ME and
  impact assessment
• As a result, contributing to project and program
  adaptive management and thus likelihood of
  impact

5
History and Current Work

• Past Work in Nigeria on Striga
• Douthwaite, B., T. Kuby, E. van de Fliert and S.
  Schulz. 2003. Impact Pathway Evaluation An
  approach for achieving and attributing impact in
  complex systems. Agricultural Systems 78
  pp243-265
• Douthwaite, B., Schulz, S., Olanrewaju, A.,
  Ellis-Jones, J. 2006. Impact pathway evaluation
  of an integrated Striga hermonthica control
  project in Northern Nigeria. Agricultural
  Systems. Published on-line
• Current Work (since Oct. 2005)
• CPWF-supported, CIAT-led impact assessment
  project in 9 river basins (900,000)
• Douthwaite, B., Alvarez, B.S., Cook, S., Davies,
  R., George, P., Howell, J and Mackay, R. 2006.
  The Impact Pathways Approach A Practical
  Application of Program Theory in
  Research-for-Development. For submission to an
  Evaluation Journal
• Future Work
• EU-funded, Wageningen-led eco-system approach
  for co-innovation of farm livelihoods project
  (Euro 1.8 million with 6 PhDs and 4 PostDocs)
• Phase II of Knowledge Sharing for Research
  Project (with Simone Staiger)
• PRGA INIS Project (CIAT and CIP led)
• See www.impactpathways.pbwiki.com

6
Impact pathways two conceptualizations.
Logic model
Network maps
7
Impact Pathways

• Two complementary conceptualizations of a
  project, a program or an organizations impact
  pathways
• A visual description of the causal chain of
  events and outcomes that link outputs to the goal
  (logic model) and
• Network maps that show the evolving relationships
  necessary to achieve the goal
• Implementing organizations stakeholders
  ultimate beneficiaries
• Shows the project rationale its logic
• Foundation of ex-ante (and ex-post) impact
  assessment

8
Foundations of the IP Approach

• Synthesis of concepts and tools from
• Program Evaluation
• Renger and Titcomb (2002) problem trees
• Chen (2005) program theory
• Mayne (2004) - performance stories
• Innovation histories
• Douthwaite and Ashby, 2005
• Appreciative Inquiry
• Whitney and Trosten-Bloom, 2003
• Social network analysis
• Cross and Parker, 2004

9
The Process of Developing Impact Pathways
10
Participatory Development of Impact Pathways
11
Example of a Problem Tree
12
Turning a problem tree into an objective tree
13
The Process of Developing Impact Pathways The
Workshop
14
(No Transcript)
15
Scaling Out and Scaling Up

• Scaling up - an institutional expansion, from
  adopters and their grassroots organizations to
  policy makers, donors, development institutions
• Scaling out - spread of a project outputs (i.e.,
  a new technology, a new strategy, etc.) from
  farmer to farmer, community to community, within
  the same stakeholder groups

16
Develop a vision of project success two years
after the project ends

• Work in project groups
• Take 5 minutes to individually answer the
  question
• You wake up 2 years after your project has ended.
  Your project has been a success and is well on
  its way to achieving its goal. Describe what
  this success looks like to a journalist
• What is happening differently now?
• Who is doing what differently?
• What have been the changes in the lives of the
  people using the project outputs, and who they
  interact with?
• How are project outputs disseminating
  (scaling-out)?
• What political support is nurturing this spread
  (scaling-up)? How did that happen?
• Discuss and develop a common vision

Keep it realistic
Workshop
17
Example of a Vision
18
The Process of Developing Impact Pathways The
Workshop
19
Develop a project timeline from when your project
started until 2 years after it will end

• Build a timeline of activities, outputs and
  outcomes that take you from the beginning of the
  project to achieving the vision
• It is a story of adoption of project outputs
  (scaling-out) and the political support that
  helps it along (scaling-up)

Workshop
20
Example of a Timeline
21
The Process of Developing Impact Pathways The
Workshop
22
Family ties Friendship ties Workplace ties
23
Todays tasks..

• Identify relevant actors relationships
• Develop network diagrams for
• Your project now
• Residual network 2 years after project has
  finished
• Identify key extension (scaling out) and
  political support (scaling up) linkages
• Identify differences between the two networks and
  discuss implications

Workshop
24
(No Transcript)
25
Differences between the MUS projects maps
26
The Process of Developing Impact Pathways After
the Workshop
27
Impact Narrative

• Text description of the project impact pathways
• Achieves the integration between the logic and
  network models
• Helps with colligation (tracing of logical steps,
  Roberts, 1996), making hidden assumptions
  explicit
• Helps with the plausibility of ex-ante impact
  assessment

28
IP Logic Model (Draft 1)
29
IP Logic Model (Final)
30
Network Maps
31
Separate relationships
32
Final maps based on answers
33
The theory behind the IP approach
Stakeholders' implicit theories are not likely
to be systematically and explicitly articulated,
and so it is up to evaluators to help
stakeholders elaborate their ideas. (Chen, 2005,
p. 14)
34
Impact of IPs 1

• From Workshops
• I will use Impact Pathways in future design of
  projects
• The dynamics of the networks is useful to
  envision the future
• It helps show gaps
• It is good for planning
• It helps explain impact of my project
• Constructing impact pathways should not be
  one-shot
• The impact pathways should be a living document

35
Impact of IPs 2

• Significant Change Stories resulting the Volta IP
  Workshop
• Locally-organized IP Workshop to build basin
  program
• Exploiting an opportunity for political lobbying
• Network concepts hybridized with influence
  mapping to become main PhD research methodology
• Problem and objective trees used to explain
  project to primary stakeholders

36
Impact of IPs 3

• Science Council review of CPWF 2007 2009 Medium
  Term Plan
• The CPWF has introduced the use of objective
  trees at the MTP project and CP level, a useful
  and innovative complement to the MTP logframe.
  In addition to providing a useful overview, the
  process of preparing these flow charts has
  clearly helped the CP provide the necessary
  focus, clarity and cohesion that now exists in
  the research plans at all levels.

37
Impact Pathways Evaluation

• Monitoring and evaluating progress along impact
  pathways
• Regularly updating objective tree, timeline and
  network maps
• Most Significant Change to pick up unexpected
  consequences
• Provides the information needed for adaptive
  management
• Impact Pathways Evaluation Action research
• Is publishable raises the status of ME

38
IPs and ex-post Impact Assessment

• Ex-post impact assessment should identify and
  describe (EIARD, 2003)
• The concept or model of innovation
• The logic model underlying a project or program
• In other words, the normative and causative
  program theory

39
Research Questions

• What types of network should R4D programs attempt
  to foster to achieve impact?
• Food security, poverty alleviation, improved
  health, environmental security
• How does improving the congruence between
  implicit and explicit stakeholder theories
  improve research-for-development effectiveness?
• Are impact pathways generalizable?
• Can one projects impact pathways help planning
  and implementation of others?

40
IPs the elevator conversation

• What do you do?
• Work to mainstream an impact orientation in
  agricultural R4D projects and programs
• What is impact orientation?
• That the people implementing projects are clearer
  about how their research will make a difference,
  and take responsibility that it does
• How do you do that?
• Work with practitioners to make explicit their
  impact pathways
• Carry out research to understand which impact
  pathways work, when

41
Summary

• Change Models matter People plan and implement
  projects on the basis of their change models -
  their implicit theories about how the world works
• If you can improve these theories you can improve
  the practice, making impact more likely
• The Impact Pathways Approach A participatory
  approach for
• Making practitioners theories explicit about how
  they will achieve adoption and impact (impact
  pathways)
• Improving these theories
• Using these models / frameworks for ME and
  impact assessment
• IP Evaluation Action research
• As a result, contributing to project and program
  adaptive management and thus likelihood of
  impact
• A work in progress, including research on network
  structures and impact pathways

42
Impact Pathways Matter