Title: Impact Pathways: An Approach for Understanding, Fostering and Evaluating Research-for-Development Outcomes
1Impact Pathways An Approach for Understanding,
Fostering and Evaluating Research-for-Development
Outcomes
- Boru Douthwaite, Technology Policy Analyst
- Sophie Alvarez, Consultant
- Simon Cook, Leader BFPs
- Rick Davies, ME Specialist,
- Pamela George, CPWF Program Manager,
- John Howell, ME Specialist,
- Ronald Mackay, Professor Emeritus,
- Jorge Rubiano, National University of Colombia
- CIAT Seminar, 1st November 2006
2Impact Pathways Matter
3How change happens
- Improvements in poverty alleviation, food
security and the state of natural resources
result from dynamic, interactive, non-linear, and
generally uncertain processes of innovation. - EIARD, 2003
- EIARD represents a group of European donors
- 15 EU Countries plus Norway and Switzerland
4Impact Pathways Approach
- People plan and implement projects on the basis
of their change models - their implicit theories
about how the world works - If you can improve these theories you can improve
the practice, making impact more likely - Impact Pathways Approach A participatory
approach for - Making practitioners theories explicit about how
they will achieve adoption and impact (impact
pathways, program theory) - Improving these theories
- Using those models / frameworks for ME and
impact assessment - As a result, contributing to project and program
adaptive management and thus likelihood of
impact
5History and Current Work
- Past Work in Nigeria on Striga
- Douthwaite, B., T. Kuby, E. van de Fliert and S.
Schulz. 2003. Impact Pathway Evaluation An
approach for achieving and attributing impact in
complex systems. Agricultural Systems 78
pp243-265 - Douthwaite, B., Schulz, S., Olanrewaju, A.,
Ellis-Jones, J. 2006. Impact pathway evaluation
of an integrated Striga hermonthica control
project in Northern Nigeria. Agricultural
Systems. Published on-line - Current Work (since Oct. 2005)
- CPWF-supported, CIAT-led impact assessment
project in 9 river basins (900,000) - Douthwaite, B., Alvarez, B.S., Cook, S., Davies,
R., George, P., Howell, J and Mackay, R. 2006.
The Impact Pathways Approach A Practical
Application of Program Theory in
Research-for-Development. For submission to an
Evaluation Journal - Future Work
- EU-funded, Wageningen-led eco-system approach
for co-innovation of farm livelihoods project
(Euro 1.8 million with 6 PhDs and 4 PostDocs) - Phase II of Knowledge Sharing for Research
Project (with Simone Staiger) - PRGA INIS Project (CIAT and CIP led)
- See www.impactpathways.pbwiki.com
6Impact pathways two conceptualizations.
Logic model
Network maps
7Impact Pathways
- Two complementary conceptualizations of a
project, a program or an organizations impact
pathways - A visual description of the causal chain of
events and outcomes that link outputs to the goal
(logic model) and - Network maps that show the evolving relationships
necessary to achieve the goal - Implementing organizations stakeholders
ultimate beneficiaries - Shows the project rationale its logic
- Foundation of ex-ante (and ex-post) impact
assessment
8Foundations of the IP Approach
- Synthesis of concepts and tools from
- Program Evaluation
- Renger and Titcomb (2002) problem trees
- Chen (2005) program theory
- Mayne (2004) - performance stories
- Innovation histories
- Douthwaite and Ashby, 2005
- Appreciative Inquiry
- Whitney and Trosten-Bloom, 2003
- Social network analysis
- Cross and Parker, 2004
9The Process of Developing Impact Pathways
10Participatory Development of Impact Pathways
11Example of a Problem Tree
12Turning a problem tree into an objective tree
13The Process of Developing Impact Pathways The
Workshop
14(No Transcript)
15Scaling Out and Scaling Up
- Scaling up - an institutional expansion, from
adopters and their grassroots organizations to
policy makers, donors, development institutions - Scaling out - spread of a project outputs (i.e.,
a new technology, a new strategy, etc.) from
farmer to farmer, community to community, within
the same stakeholder groups
16Develop a vision of project success two years
after the project ends
- Work in project groups
- Take 5 minutes to individually answer the
question - You wake up 2 years after your project has ended.
Your project has been a success and is well on
its way to achieving its goal. Describe what
this success looks like to a journalist - What is happening differently now?
- Who is doing what differently?
- What have been the changes in the lives of the
people using the project outputs, and who they
interact with? - How are project outputs disseminating
(scaling-out)? - What political support is nurturing this spread
(scaling-up)? How did that happen? - Discuss and develop a common vision
Keep it realistic
Workshop
17Example of a Vision
18The Process of Developing Impact Pathways The
Workshop
19Develop a project timeline from when your project
started until 2 years after it will end
- Build a timeline of activities, outputs and
outcomes that take you from the beginning of the
project to achieving the vision - It is a story of adoption of project outputs
(scaling-out) and the political support that
helps it along (scaling-up)
Workshop
20Example of a Timeline
21The Process of Developing Impact Pathways The
Workshop
22Family ties Friendship ties Workplace ties
23Todays tasks..
- Identify relevant actors relationships
- Develop network diagrams for
- Your project now
- Residual network 2 years after project has
finished - Identify key extension (scaling out) and
political support (scaling up) linkages - Identify differences between the two networks and
discuss implications
Workshop
24(No Transcript)
25Differences between the MUS projects maps
26The Process of Developing Impact Pathways After
the Workshop
27 Impact Narrative
- Text description of the project impact pathways
- Achieves the integration between the logic and
network models - Helps with colligation (tracing of logical steps,
Roberts, 1996), making hidden assumptions
explicit - Helps with the plausibility of ex-ante impact
assessment
28IP Logic Model (Draft 1)
29IP Logic Model (Final)
30Network Maps
31Separate relationships
32Final maps based on answers
33The theory behind the IP approach
Stakeholders' implicit theories are not likely
to be systematically and explicitly articulated,
and so it is up to evaluators to help
stakeholders elaborate their ideas. (Chen, 2005,
p. 14)
34Impact of IPs 1
- From Workshops
- I will use Impact Pathways in future design of
projects - The dynamics of the networks is useful to
envision the future - It helps show gaps
- It is good for planning
- It helps explain impact of my project
- Constructing impact pathways should not be
one-shot - The impact pathways should be a living document
35Impact of IPs 2
- Significant Change Stories resulting the Volta IP
Workshop - Locally-organized IP Workshop to build basin
program - Exploiting an opportunity for political lobbying
- Network concepts hybridized with influence
mapping to become main PhD research methodology - Problem and objective trees used to explain
project to primary stakeholders
36Impact of IPs 3
- Science Council review of CPWF 2007 2009 Medium
Term Plan - The CPWF has introduced the use of objective
trees at the MTP project and CP level, a useful
and innovative complement to the MTP logframe.
In addition to providing a useful overview, the
process of preparing these flow charts has
clearly helped the CP provide the necessary
focus, clarity and cohesion that now exists in
the research plans at all levels.
37Impact Pathways Evaluation
- Monitoring and evaluating progress along impact
pathways - Regularly updating objective tree, timeline and
network maps - Most Significant Change to pick up unexpected
consequences - Provides the information needed for adaptive
management - Impact Pathways Evaluation Action research
- Is publishable raises the status of ME
38IPs and ex-post Impact Assessment
- Ex-post impact assessment should identify and
describe (EIARD, 2003) - The concept or model of innovation
- The logic model underlying a project or program
- In other words, the normative and causative
program theory
39Research Questions
- What types of network should R4D programs attempt
to foster to achieve impact? - Food security, poverty alleviation, improved
health, environmental security - How does improving the congruence between
implicit and explicit stakeholder theories
improve research-for-development effectiveness? - Are impact pathways generalizable?
- Can one projects impact pathways help planning
and implementation of others?
40IPs the elevator conversation
- What do you do?
- Work to mainstream an impact orientation in
agricultural R4D projects and programs - What is impact orientation?
- That the people implementing projects are clearer
about how their research will make a difference,
and take responsibility that it does - How do you do that?
- Work with practitioners to make explicit their
impact pathways - Carry out research to understand which impact
pathways work, when
41Summary
- Change Models matter People plan and implement
projects on the basis of their change models -
their implicit theories about how the world works - If you can improve these theories you can improve
the practice, making impact more likely - The Impact Pathways Approach A participatory
approach for - Making practitioners theories explicit about how
they will achieve adoption and impact (impact
pathways) - Improving these theories
- Using these models / frameworks for ME and
impact assessment - IP Evaluation Action research
- As a result, contributing to project and program
adaptive management and thus likelihood of
impact - A work in progress, including research on network
structures and impact pathways
42Impact Pathways Matter