Assessing the Relevance of Global and Regional Partnership Programs (GRPPs)

1
Assessing the Relevance of Global and Regional
Partnership Programs (GRPPs)

• Chris Gerrard
• Global Programs Coordinator, IEG
• November 13, 2009
• http//www.globalevaluations.org

2
Main Messages

• Need to assess the beneficiary demand for the
  program as well as donor supply of financial
  resources
• Need to assess the programs value added both
  vertically and horizontally in relation to the
  partners own activities as well as to other
  global initiatives
• Need to assess relevance of the programs design
  in addition to the relevance of its objectives

3
Special Features of GRPPs in Relation to
Relevance

• Often arise out of international conventions,
  conferences, and meetings
• Most have been initiated by donors
• Usually conduct activities at multiple levels
  global, regional, national and local
• Often compete with other programs and activities
  in the same sector
• Positive outcomes also depend on complementary
  activities other development partners and
  country-level stakeholders

4
Assessing Relevance -- Suggested Dimensions

• Supply-side relevance The existence of an
  international consensus that global/regional
  collective action is required.
• Demand-side relevance Consistency with the
  needs, priorities, and strategies of beneficiary
  countries.
• Vertical relevance (or subsidiarity) Not
  competing with the major partners own
  development assistance activities
• Horizontal relevance The absence of alternative
  sources of supply of the same goods and services.
  
• Relevance of the design The appropriateness of
  the strategies and priority activities of the
  program for achieving the objectives of the
  program.

5
Assessing Relevance Suggested Approach

• First of all, come to some understanding on the
  (higher order) goals, (immediate) objectives,
  strategies and major activities of the program,
  and how these have changed over time
• Not always clearly articulated in historical
  program documents or in the evaluation TOR
• Often some implicit objectives (which may exist
  for strategic or other reasons)
• May need to construct at least a rudimentary
  results framework in order to assess relevance of
  design
• Then use an evidence-based analytical approach to
  assess the objectives against the five
  dimensions, relative to the time at which the
  evaluation is taking place.

6
Extent to Which 60 Evaluations Assessed Different
Dimensions of Relevance
7
Good Practice Examples Supply-Side Relevance

• Global Donor Platform for Rural Development
  (GDPRD)
• Program Objectives To improve donor cooperation/
  collaboration and coordinated dialogue with
  partner countries
• Evaluation Methodology Analysis of international
  context in which the program operates,
  interviews, and stakeholder analysis
• Evaluation Findings
• Objectives are congruent with international
  consensus for improved donor coordination (e.g.
  Paris Declaration)
• But there is a lack of consensus among
  stakeholders on the definition of the problem, or
  role of the Platform in addressing it
• Recommendation GDPRD Board should initiate a
  strategic process to clarify the Platforms role,
  including the preparation of strategic and
  operational business plans to guide its future
  development

8
Demand-Side Relevance

• Public-Private Infrastructure Advisory Facility
  (PPIAF)
• Program Objectives To help facilitate
  public-private partnerships to improve countries
  infrastructure
• Evaluation Questions
• Is there a sustainable demand for PPIAF services
  across developing countries?
• Are beneficiary country priorities reflected in
  the distribution of PPIAFs services?
• Methodology Portfolio analysis of individual
  grants in relation to country priorities
• Findings
• PPIAF is demand-driven by design, but filtered
  through WB task managers
• Types of services provided generally reflect
  recipient government priorities
• Recommendation PPIAF should give greater
  priority to those countries with clear national
  policies and strategies for infrastructure
  development

9
Vertical Relevance -- Subsidiarity

• Global Alliance for Vaccines and Immunization
  (GAVI)
• Program Objectives To increase access to
  immunization in poor countries
• Evaluation Question The extent to which GAVI was
  competing or substituting for the activities of
  its major partners (WHO, UNICEF).
• Findings
• GAVI has achieved better immunization outcomes
  than its partners could have achieved
  individually by (a) engaging players in the
  field, (b) raising global immunization funding to
  an unprecedented level, and (c) applying
  innovative approaches to address urgent
  immunization problems
• GAVI has been less successful in advocating the
  immunization agenda at the country level
• Recommendation GAVI should accelerate its
  efforts to build stronger country ownership and
  commitment to the immunization agenda.

10
Horizontal Relevance Absence of Alternative
Sources of Supply

• Water and Sanitation Program (WSP)
• Program Objectives To improve access of the poor
  to improved water and sanitation services.
• Evaluation Methodology Detailed analysis of the
  value added of WSP in relation to other global
  programs in the water sector , namely, WSSCC and
  GWP
• Findings Each of these three programs has a
  distinct role in the sector which is
  complementary, but the programs are working
  largely independently of each other
• Recommendation WSP should improve its working
  relationships with these other programs, and
  explore avenues of improved cooperation to
  achieve shared objectives.

11
Relevance of Design

• International Center for Tropical Agriculture
  (CIAT)
• Program Objectives Collaborative research to
  improve agricul-tural productivity natural
  resource management in the tropics.
• Evaluation Methodology In-depth analysis of
  CIATs research activities.
• Findings A large number of its research
  activities were not clearly linked to its
  objectives, creating inefficiencies in deploying
  its resources and reducing its development
  impact.
• Recommendation CIAT should revise its strategic
  plan around small system-based outcome lines
  rather than separate activities/products and
  concentrate more on regional as opposed to global
  level research.

12
Overall Conclusions

• Need to confirm that the international consensus
  underlying the program extends beyond the donors
  to include the intended beneficiaries
• Need to assess the value added both vertical
  and horizontal of those programs that are not
  providing global or regional public goods
• Need to pay particular attention to program
  design. Few programs have a well-articulated
  theory of change from inputs and outputs to
  desired outcomes and impacts along with
  measurable performance indicators.

13
Thank You