Authors: Naman Keita (FAO), Nwanze Okidegbe, Sanjiva Cooke (World Bank), Tim Marchant, Consultant

1
Session 3 Topic 3 Developing countries
perspective Selecting a core set of Indicators
for Monitoring and Evaluation in Agriculture and
Rural Development in Less-than-Ideal Conditions
and implications for countries statistical system
WYE CITY GROUP MEETING ON STATISTICS ON RURAL
DEVELOPMENT AND AGRICULTURE HOUSEHOLD INCOME ,
Rome 11-12 June 2009

• Authors Naman Keita (FAO), Nwanze Okidegbe,
  Sanjiva Cooke (World Bank), Tim Marchant,
  Consultant
• Presented by N. Keita, Senior Statistician, FAO

World Bank
2
PRESENTATION OUTLINE

• Agriculture and Rural Development Policy Issues
  in Developing Countries and ME framework for
  tracking Results
• The analytical framework
• The Indicators
• The data framework
• Capacity of National Statistical Systems

3
1. ARD POLICY ISSUES AND ME FRAMEWORK

• ?Developed countries agriculture is less and
  less the economic base of rural areas.
• ?Developing countries agricultural base of the
  economy with
• 40 of the workers
• over 20 of their GDP
• around 75 of the poor still live in rural areas
  and
• the proportion of rural population to total
  population is comprised between 59.5 in less
  developed regions in 2000 (estimate of 56.8 in
  2005) and 74.8 in least developed countries
  (72.3 in 2005)
• ?The major policy issues are related to
• sustainable agriculture and rural development and
  
• long term improvement of the peoples living
  standard, particularly the rural population
  including food security

4
1. ARD POLICY ISSUES AND ME FRAMEWORK

• Sector-wide approach (SWAP) to ARD Programmes ?
  growing demand for verifiable evidence of the
  results and impacts of development programs.
• Most indicators are focused on performance and
  relate mainly to inputs and outputs (used to
  populate management information systems).
• Results indicators have become increasingly
  prominent in the wake of recent international
  resolutions such as the Paris Declaration on Aid
  Effectiveness in 2005 and the Monterrey Consensus
  on Financing for Development in 2002.
• Emphasis on aid effectiveness and results-based
  development ? need to demonstrate the impacts of
  their projects and programs ? shifted the focus
  of ME from a concentration on inputs and outputs
  to a concentration on outcomes and impacts.
• To measure outcomes and impacts imply the use of
  indicators that are based on reliable data, and
  on the capacity to systematically collect and
  analyze that information.
• In most developing countries conditions are
  less-than-ideal. Information is irregular and
  often lacking altogether .
• Strengthening capacity for ME begins at the
  national and sub-national levels, where
  addressing the weaknesses of national statistical
  systems is a common priority.

5
1. ARD POLICY ISSUES AND ME FRAMEWORK

• The ideal environment for establishing a good
  ME system is where
• there is a strong and consistent demand for
  information.
• the concept of "management by results" is widely
  practised.
• timely and relevant information is being
  systematically used to improve decision-making
  and to advance the process of development.
• systems are in place to ensure that reliable and
  relevant data and information are available when
  needed.

• The less-than-ideal condition, on the other
  hand, is where
• demand for information is weak.
• evidence is not used to inform decision-making.
• the stock and flow of timely information are
  irregular and unreliable and statistical capacity
  is weak.

6
2. ANALYTICAL FRAMEWORK

• Logframe
• Tracking inputs and outputs
• Public Expenditure Tracking System (PETS) and
  Quantitative Service Delivery Survey (QSDS)
• Focus of this Sourcebook on
• Measuring results (outcomes and impact)
• Early outcomes
• Later outcomes
• Defining a core set of priority indicators for
  ARD programs

7
3. INDICATORSDifficulties with the measurement
of agricultural output
8
3. INDICATORSFrequency of Monitoring various
Indicators
Medium to Long Run (maybe 5 years by the time
surveys are carried out)
Impact Indicators (Ultimate goal)
Outcome Indicators (behavioral change)
Medium Term - Ideally annually maybe every 2
3 years
Output Indicators (Goods and Services)
Short Medium Term - Ideally more than once a
year or annually
Input Indicators (Material, financial, human)
Short Term - Ideally every three months or
annually
Issues Cost and Capacity
9
3. INDICATORSWhat makes a Good Indicator?

• S SPECIFIC and SENSITIVE to the changes induced
  as a result of actions taken
• M MEASURABLE progress can be shown and is not
  easily manipulated
• A ATTAINABLE and APPLICABLE to the policy action
  taken
• R RELEVANT to the areas in question
• T TIME BOUND and TRACKABLE by showing changes
  over time
• Could also be RAVES
• Reliable
• Appropriate
• Valid
• Easy to collect
• Sensitive and specific

10
3. INDICATORS COUNTRY VALIDATIONS

• Countries Cambodia, Nicaragua, Nigeria, Senegal,
  Tanania
• Purpose Test the conceptual framework and a
  preliminary list of indicators against country
  capacity (ME and Statistics), practice and learn
  lessons
• Findings
• All countries are engaged in strengthening and
  rationalising the national ME System in parallel
  with Statistical reform (NSDS)
• Countries are at different stages regarding ME
  and statistical development (Examples of Senegal
  and Tanzania)
• Disconnect between ME and Statistical System
• Outcome Revised list of indicators, framework
  and good practices
• Emerging Issues and challenges
• Decentralization and devolutiongtimplication for
  M7E and Stat system
• Linkage and articulation between ME and
  Statistics system
• Linkages with international agencies

11
3. INDICATORS Results of the country validation
studies
Subsector Total indicators No. of generic indicators currently available No. of generic indicators currently available No. of generic indicators currently available No. of generic indicators currently available No. of generic indicators currently available No. of generic indicators currently available No. of generic indicators currently available
Subsector Total indicators Cambodia Nicaragua Nicaragua Nigeria Senegal Senegal The United Republic of Tanzania
A. Core ARD sector indicators 28 8 7 9 9 8 3 3
B. Agribusiness and market development 13 2 4 4 4 3 3 3
C. Community-based rural development 9   2 4 4   2 2
D. Fisheries (aquaculture) 6 3 3 1 1 1    
E. Forestry 13 5 3 3 3 5 3 3
F. Livestock 8 5 5 7 7 6 2 2
G. Policies and institutions 18 6 11 11 11 7 6 6
H. Research and extension 7 4 3 4 4      
I. Rural Finance 7   5 5 5   4 4
J. Sustainable land and crop management 9 6 6 5 5 2    
K. Water resource management 13 1 7 3 3 6 4 4
Total    131 40 56 56 56 38 27 27
12
Arranged bySub-sectors and thematic areas A.
Sector-Wide Indicators for Agriculture and Rural
Development B. Specific Indicators for
Sub-sectors of Agriculture and Rural
Development (1-Crops, 2-Livestock, 3-Fisheries
and Aquaculture, 4-Forestry, 5-Rural Micro and
SME Finance, 6-Agriculture Research and
Extension, 7-Irrigation and Drainage,
8- Agri-Business) C. Indicators for Thematic
Areas related to Agriculture and Rural
Development (1-Community-based rural
development, 2-Natural Resources Management,
3-Land Policy and Administration)Early
outcome and long-term indicators
3. INDICATORS MENU of 86 indicators
13
3. INDICATORSNineteen priority indicators(MAIN
CRITERIA USED RELEVANCE, COMPARABILITY,
AVAILABILITY)
1 Public spending on agriculture as a of GDP from agriculture sector 11 of the rural population using financial services of formal banking institutions
2 Public spending on agricultural input subsidies as a percentage of total public spending on agriculture 12 Public investment in agricultural research as a of GDP from agriculture sector
3 Prevalence () of underweight children under five years of age in rural areas 13 Irrigated land as a of crop land
4 Food production index 14 change in sales/ turnovers of agro-enterprises
5 annual growth in agricultural value added 15 of farmers who are members of producer organizations
6 Rural poor as a proportion of total poor population 16 Agricultural withdrawal as of total freshwater withdrawal  
7 change in yields of major crops of the country 17 Proportion () of land area formally establishes as protected area
8 annual growth in value added in the livestock sub-sector 18 change in soil loss from watersheds
9 Capture fish production as of existing stock ( or a rating of state of major capture fish stocks relevant to exports and local food) 19 land area for which there exists a legally recognized form of land tenure
10 Proportion of land area covered by forest () (see menu of 86 indicators.doc)
14
4. DATA FRAMEWORK Sources of data
Inputs Outputs Outcomes Impacts
Administrative information systems
Survey Programme of the CSO or MoA (Population census, Agricultural census, LSMS, DHS..)
Participatory Poverty Assessments
Financial Management Tools
15
4. DATA FRAMEWORK TOOLSSurveys vs. non-formal
appraisal methods

16
4. DATA FRAMEWORK

• Applying the tools for ME analysis
• Comparisons over time
• Baseline surveys
• Panel surveys
• Comparisons over space
• Counterfactual comparisons (with and without)
• Does the National Statistical System have the
  capacity to deliver?

17
4. DATA FRAMEWORK
Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys Comparison of key features of different surveys
1 2 3 4 5 Best used for Best used for Best used for Best used for
  Sample size Duration Visits to household Question-naire size Cost (m) Time series Sub- nat'l Counter- factual Counter- factual
Population census Full coverage 3-6 months 1 4-8 15-25 ? ?? ? ?
Agricultural census 20 000-50 000 1-1.5 years 2-4 5-12 5-10 ? ?? ? ?
LSMS/integrated survey 5 000-10 000 1-1.5 years 2 40 1-2 ? ? ?? ??
Household budget survey 4 000-10 000 1-1.5 years 15-25 15-20 1-2 ? ? ?? ??
Community survey 100-500 4-6 months 1 4-6 0.2-0.4 ? ? ? ?
Service delivery survey (CWIQ) 10 000-15 000 2-3 months 1 8 0.2-0.4 ?? ? ? ?
Focus group interviews 40-50 2-3 months 1-3 - 0.05-0.1 ? ? ? ?
Windscreen survey 10-20 2-3 weeks 0   0.01 ? ? ? ?
?not suitable ?adequate ??good
18
5. CAPACITY OF NATIONAL STATISTICAL SYSTEMS

• Main problems common to many developing
  countries
• limited staff and capacity of the units that are
  responsible of for collection, compilation,
  analysis and dissemination of agricultural
  statistics
• lack of adequate technical tools, packages and
  framework to support countries data production
  efforts
• insufficient funding allocated of agricultural
  statistics from development partners and national
  budget
• lack of institutional coordination which results
  in the co-existence of not harmonised and
  integrated data sources
• lack of capacity to analyse data in a policy
  perspective which results in a significant waste
  of resources as large amounts of raw data are not
  properly used
• difficult access to existing data by users with
  no metadata and indication of quality

19
5. CAPACITY OF NATIONAL STATISTICAL
SYSTEMSOpportunities

• ? growing interest in the monitoring and
  evaluation of national development
  programmesgtgrowing interest in the
  rehabilitation of the NSS
• ?NSDS process
• ?Global strategy will provide
• the framework to integrate a core set of
  agricultural and rural statistics into the
  national and international statistical systems,
• identify a suite of methodologies for the data
  collection, provide a framework for integrating
  agricultural and rural statistics with the
  overlapping data requirements of other sectors,
  and address the need to improve statistical
  capacity.
• propose a governance structure for coordination
  not only between the national statistical
  organisations and other country ministries, but
  also between national statistical organisations
  of other countries, donors, and regional and
  international organisations.
• ?Global Strategy to be discussed by senior
  experts during the upcoming International
  Statistical Institute Satellite meeting to be
  held 13-14 August 2009 in Maputo, Mozambique

20
TRACKING RESULTS IN AGRICULTURE AND RURAL
DEVELOPMENT IN LESS-THAN-IDEAL CONDITIONS

• A Sourcebook of indicators for monitoring and
  evaluation

World Bank
21
Thank You!