Participatory Monitoring and Evaluation for strengthening institutional and community learning and c

1
Participatory Monitoring and Evaluation for
strengthening institutional and community
learning and change processes
Jemimah Njuki, Susan Kaaria, Colletah Chitsike
and Pascal Sanginga
2
Outline

• Background and Introduction
• Evaluation methods and evolution of PME
• PME in CIAT
• Methodology
• Key steps in the PME process
• Results
• Multiple uses of PME
• Conclusions and key issues

3
Evolution of evaluation methods

• ME practice emerged from the focus the Logical
  Framework Approach (USAID during the 1970s) as a
  tool for project planning, monitoring, evaluation
  and impact analysis
• Fourth Generation Evaluation which emphasized on
  involvement of various stakeholders in the
  process (GUBA and LINCOLN 1989)
• Utilization-focused Evaluation which emphasized
  that evaluations should be judged by their
  utility and use by intended users (PATTON 1978,
  1986, 1997)
• Empowerment Evaluation, which emphasizes use of
  evaluation concepts and techniques to foster
  self-determination. Through self-assessment and a
  groups knowledge of itself, it achieves
  accountability onto itself as well as to others
  (FETTERMAN 1993)
• More recently, beneficiary involvement in
  monitoring and evaluation, participatory ME, is
  viewed as a logical extension of the other
  dimensions of participation (e.g. Uphoff 1991
  Germann et. al. 1996 UNDP 1997 Abbot and Guijt
  1998 Estrella and Gaventa 1998 Estrella et al.
  2000)

4
Why interest in PME?

• Whether PME is actually fulfilling the
  expectations of community empowerment and
  learning as well as promoting institutional
  learning and change
• Understanding how PME should be structured to
  promote these community and institutional
  functions
• Information on what has worked is limited
  especially in CD-PME systems
• Integrating reflection processes to look back and
  plan ahead-- learning from experience and to
  enhance the understanding of innovation processes
• Using results of monitoring and evaluation into
  our plan of action and in other projects

5
PME in CIAT

• Objectives
• Analyze the role of PME in promoting
  institutional learning and change and enhancing
  performance and impact of projects
• Analyze the role of community PME in empowering
  communities, allowing them to analyze and
  interpret change, to learn from their own
  experiences, and to adjust strategies
• Develop tools, methods, strategies / principles
  for involving different stakeholders in the PME
  process

6

• Two levels of PME
• Project / Institutional level PME
• 2 CIAT projects
• Enabling Rural Innovation ( 3 countries in Africa
  and expanding)
• Pan African Bean Research Alliance (17 countries
  in Africa)
• Kenya Agricultural Research Institute (10
  projects)
• Community Level PME
• 30 farmer groups in 3 countries (Kenya, Uganda
  and Malawi)

7
Methodology Steps in the PME process
1. Identifying and engaging stakeholders
2. Building stakeholders capacity for PME
3. Defining and agreeing on what to monitor and
evaluate
4. Developing and formulating indicators
Reflection, sharing and using the results of PME
5. Gathering information
6. Managing analyzing data
8. Learning and change
8
Methodology Key issues in the PME process

• Engaging stakeholders and building capacity for
  PME
• Development of methods for building capacity for
  PME such as
• the use of simple graphics at community level
• Use of drama, role play and visualization tools
• Manuals and guides, general principles
• Understanding the implications of engaging
  different stakeholders in the process
• Negotiation
• Diversity of perspectives
• Diversity of tools and methods

9
Defining and agreeing on what to monitor and
evaluate
What do we want to achieve? What key changes do
we expect? What are the key activities to achieve
these changes?
Workshop and mentoring process for project
teams Use of visualization tools and role plays
for communities and local stakeholders
10
Agreeing on what to monitor
Reach

• Impacts are long term developmental results at
  the societal level linked to the goal or vision.
  combination of outputs and outcomes, but are not
  the direct results of a single project or
  program. E.g.. Improved livelihoods, poverty
  eradicated..
• Outcomes are medium term, end of project
  developmental results
  benefiting an identified
  target population that are
  achievable within the
  time frame of the project.
• E.g. Organizational change
• Activity Conduct training

ULTIMATE IMPACT
OUTCOME
Processes (partnerships, context of the
implementation process, social capital, group
organization etc)
OUTPUT

• Outputs are the short term, immediate, visible
  and concrete and tangible developmental results
  that are the immediate consequences of project
  inputs and activities.
• E.g. Increased capacities of partners to apply
  methods

ACTIVITIES
Time
11
Using Visualization to develop a community vision
of change The River Code
Desired Future Situation Where do we want to be?
Current Situation Where are we now?
Objectives/ Goals
Baseline
The River
Steps for Crossing
One side of the river
The other side of the river
12

• Developing indicators
• Combining qualitative and quantitative indicators
• Local and scientific indicators
• Integrating gender
• Targeted Baselines
• Data collection and analysis
• Sharing roles and responsibilities
• Simplifying tools for use by local stakeholders
• Building capacity for data collection and
  analysis
• Reflection
• Critical to the PME process
• Leads to learning and change
• Analyses what is working, what is not working,
  what needs to be changed
• Need for initial facilitation of the reflection
  process

13
Examples of Community Indicators, Kitale and
Mtwapa Kenya, 2004
Results Inclusion of different stakeholder
perspectives
14
Community organization, learning and
accountability

• Diversity of tools for collecting data on
  community indicators

15
Community organization, learning .
Analyzing the results of Community level PME
data to assess trends and increase accountability
16
Community organization, learning .
17
Institutional organization and learning to
improving the project implementation and
organizational processes

• Systematization of the PME process
• Feeding results of the PME process into project
  improvement through reflection and adjustment of
  activities
• Re-orienting research programmes
• New activities
• Improving data collection and reporting
• Impact orientation of projects
• Stakeholder participation
• Information sharing process
• Focus on methodologies and approaches

18
Identifying indicators to measure empowerment

• Some results of participatory process such as
  empowerment are often hard to measure
• Developing indicators for such concepts from
  community perspectives gives insights and enables
  their realistic measurement

19

• Examples of community indicators for empowerment

20
Key issues

• Institutionalization of PME
• Integration of PME into project implementation
  process
• Reflection and learning as a way of doing
  business-issues of attitude and mind sets
• Role of facilitation and capacity building in the
  process not only in PME but also in facilitation
  skills
• The importance of a systematic process for data
  collection, analysis, reflection and learning
  while at the same time maintaining flexibility
• Knowledge management and mechanism for wider
  sharing of lessons within institutions
• Engaging in policy dialogue and sharing results
  with policy-makers both within and outside the
  institution

21
Key issues

• Bringing in different stakeholders, when to bring
  them in and negotiating different stakeholders
  interests, values and systems
• Costs of the PME process vis a vis the benefits
  to institutions and communities
• Learning how to deal with conflicts --
  difficulties in handling transparency and dealing
  with sensitive topics (corrective action does not
  follow automatically
• Scaling-up micro-level information generated at
  the community and project levels to regional and
  national levels

22
Acknowledgements

• Kenya-KARI
• Malawi-DARS, Plan Malawi
• Uganda-NARO, Africa 2000 network
• Bean Programmes in PABRA countries