Techniques for Evaluating Public Policies in Developing Countries (DCs)

1
Techniques for Evaluating Public Policies in
Developing Countries (DCs)

• Luiz Awazu Pereira da Silva
• Ministry of Finance (Brazil)
• University of Palma de Mallorca
• February 4th 2005
• Based on The Impact of of Economic Policies on
  Poverty and Distribution
• by François Bourguignon, Luiz A. Pereira da
  Silva, eds.
• The World Bank, Oxford University Press (2003)

2
Outline of this Presentation

• Policy Challenges for DCs the evaluation of
  public expenditures and economic policies from
  aggregate macro to micro distribution/poverty
• Framework for evaluating public policies
• Part 1 Microeconomic evaluation techniques
• Part 2 Macro evaluation techniques (micro-macro
  linkages)
• Future directions for more complex techniques
• Practical difficulties for DCs (institutional
  set-up)

3

• Evaluation techniques in DCs have evolved
  together with
• Development Economics, e.g., goals and theory
• Data Availability and Econometric Techniques,
  e.g., HHS, Firm data
• Modeling techniques, e.g., CBA, CGEs
• Challenges of Globalization , e.g., political
  economy in DCs
• Policy Challenges for DCs now linking the
  evaluation of public expenditures, economic
  policies to distribution/poverty

4
Policy Challenges 1950-1970s ? old vision of
Evaluation of Public Policies

1. Maximize Aggregate Growth and Minimize Risks of
   BoP crisis, under old BW international financial
   arquitecture (fixed ERs, K controls, etc.) and
   import-substituting development strategies
2. Evaluation from aggregate growth models ? level
   of external savings needed for target growth,
   Kflows (public), find the best set of projects
   doing project analysis in partial equilibrium
   (CBA) using if need be shadow-pricing
3. Most DCs with institutional structure for
   evaluation with strong Min. Planning and Project
   Analysis Unit (World Bank, IMF) and MoF in
   control of ER, BoP

5
Policy Results (1970s-1980s) of old vision of
Evaluation of Public Policies

1. Some successes but also booms and busts ? Policy
   instability, structural adjustments, external
   vulnerability
2. Fiscal and/or BoP crises ? high or hyper
   inflation, devaluations
3. Poverty and distributional challenges ? Political
   instability
4. Shift in institutional balance MoFs vs. Planning
5. Obsolescence of most project analysis units and
   of CBA in planning evaluation methods

6
Policy Challenges 1990s-2000s (1) the most
common economic policies and structural reforms
in DCs ? change in scope for Evaluation of Public
Policies

• Macro-economic policies, ST
• Fiscal monetary policy stance, exchange rate
  regime, public debt management strategy, etc.
• Public Expenditure and Revenue,
  Micro-social-policies, ST - MT
• Tax policy reform, composition of public
  expenditure, design of social programs (CCTs)
  civil service reform, pension reform,
  decentralization
• Structural Reforms, LT
• Trade liberalization, liberalization of specific
  markets, financial sector reforms, improving the
  investment climate, land reform, privatization
  etc.

7
Within point 2., in particular, most common
policy challenge for DCs is Evaluation of Public
Expenditure

• Adequate Aggregate Level ? Is Deficit, Public
  debt Sustainable?
• Definition of PS, Hidden Contingent Liabilities?
• Methodology (mechanical ratios or stochastic)?
• Adequate dynamics, counter-cyclicality of public
  spending?
• Macro policy fiscal stance and credibility
• Programs to off-set effect of volatility,
  financial crises
• Is there crowding-out or crowding in
  private/public?
• Complementarity of PE, Externalities w/ private
  sector?
• Market failures? Lobbies?
• Are allocations adequate?
• Inter-sectoral allocation, Capacity-building
• Input mix (Capital/Recurrent Wage/Non-Wage)
• What are Poverty and Distributional Impact of PE?
• Cost-Efficiency of social programs
• Outcome indicators, Evaluation methods

8
Illustration of Typical Set of Challenges for
DCs Example of Brazil

• High PS Debt and unsustainable Tax Burden ? Need
  to Generate Primary Fiscal Surpluses
• High and persistent inequality, poverty and
  Budget rigidity ? Need to Improve Targeting of
  Social Policies

9
Tax/GDP
Public Debt/GDP
Gini
10
Evaluation of Policies in DCs with these new
policy challenges broader range of micro
programs to macro policies

• Scope and objective, challenges increase
  evaluate the economic feasibility of public
  programs and policies and their overall
  development impact.
• Aggregate and first principle analysis
  insufficient heterogeneity of individuals and
  households, microeconomic behavior do not add up
  into aggregate nor average, specificities of
  economic structures and local political economy,
  transmission of shocks and policies
• Policy objectives and social demand increasingly
  focusing on distributional effects and poverty
  reduction, essentially micro concepts (e.g.,
  Post-WC IFIs, new types of Governments, etc.)
• Micro data bases (household surveys HHS)
  increasingly available as the natural analytical
  environment for distributional and poverty
  analysis
• Hence the natural idea to link the effect of
  economic policies to the corresponding changes in
  the income and/or expenditure of individuals,
  households, social groups and the poor in
  particular
• Impact evaluation allows to think about
  scaling-up and pro-poor, redistributive
  development strategies

11
Important dimensions in the evaluation of Public
Policies in DCs with these new policy challenges

• Counterfactual is needed (the world with and
  without the program or policy being evaluated,
  sometimes difficult)
• Ex-ante or ex-post (ex-ante evaluates the design
  of non-existing programs and policies, ex-post
  focus on outcomes)
• Partial or General Equilibrium (taking or not
  into account the effect of programs and policies
  on price systems and economic equilibria)
• Behavioral or Arithmetic (based or not into
  some representation of economic behavior of
  agents reacting to the program or policy )

12
Framework for Evaluation Define impact for
individual i as the difference in income yi with
and without the program, denominated Dyi
yi real income wi wage rate Li labor
supply Ei self-employment, non-wage income Ri
net transfer income Ai socio-economic
characteristics Ci consumption characteristics
? household-specific P price index p
general price index
13
Program or policy will shock one or more
components that explain the individual income yi

• A households character.

• R transfers

• p
• prices

• w wage

• L employt.

Household Survey (HHS), i individual households
Evaluation of Program and/or Economic Policy

• Compare the distribution of yP1 with the
  distribution of yP0.
• Calculate changes in inequality or poverty across
  the two distributions
• Different tools/methods differ in how they
  construct the counterfactual distribution and the
  data that are needed
• Rank results according to some agreed upon rule
  and/or objective

14
An illustration of one criteria for evaluation

• An incidence effect curve (say on income/
  expenditure changes) showing the percent change
  in per capita income of a macroeconomic policy
  (here, Indonesia financial crisis, changes in pc
  income by percentiles of the distribution)

poor
wealthy
15
Part 1 Microeconomic techniques

• 1. Average Incidence Analysis
• Tax Incidence Analysis (Sahn Younger)
• Public Expenditure Incidence Analysis (Demery)
• 2. Marginal Incidence Analysis
• Behavioral response to changes (Van de Walle)
• Poverty mapping (Lanjouw)
• 3. Impact Evaluation (randomization, matching,
  double-dif)
• a) Ex-post (Ravallion)
• b) Ex-ante (Bourguignon Ferreira)
• 4. Data and Measurement (not covered here)
• a) Multi-topic Household Surveys (Scott)
• b) Qualitative surveys (Rao Woolcock)
• c) Performance in Service Delivery (Dehn,
  Reinikka Svensson)

16
Average Incidence Analysis (Sahn Younger
Demery)

• Suitable for taxes or public expenditures.
• Aims to answer Who pays for / receives how
  much?
• Counterfactual is simply
• So that
• This is equivalent to assuming
• No behavioral response (perfectly inelastic
  demand for goods, perfectly inelastic supply of
  factors.
• Fine for marginal changes.
• But only a first order approximation to large
  taxes and/or transfers.
• No general equilibrium effects.

17
Average Incidence Analysis (Sahn Younger
Demery)

• A practical example from education expenditures
  the incidence of public spending in schooling
  category i which accrues to group j depends on
• groups js relative enrolment rates across
  schooling types i.
• Relative spending across categories i.

Once again purely arithmetic. No behavioral
response, no gen. eq. effects.
18
Indonesia, Benefit Incidence of Education
Spending, 1989

19
Benefit Incidence of Education Spending
20
Marginal Incidence Analysis(van de Walle)

• Suitable for taxes or public expenditures.
• Aims to answer How has the distribution of tax
  burden / program benefits changed in the recent
  past?
• Assumptions are less demanding than for average
  incidence analysis
• Requires either panel or repeated cross-section
  data.
• Although some have suggested using spatial
  variation in programs / taxes to proxy for
  temporal variation (Lanjouw Ravallion)

21
Poverty (and Expenditure) Maps (Lanjouw)

• Reliable poverty maps combining sample survey
  data with census data to yield predicted poverty
  rates for all households covered by the census.
•  1) Estimating Models of Consumption A model of
  consumption or standard of living using household
  survey data is estimated using the variables
  which are available both in the census and in the
  survey.
•  2) Predicting Poverty. The parameter estimates
  from the regressions (using the full household
  sample) are used to predict consumption or
  standard of living in the census data. For each
  household in the census, the parameter estimates
  from the applicable regression (conditional on
  geographical location) are combined with the
  household's characteristics in order to obtain an
  imputed value for per capita consumption
  expenditure.
• 3) Comparing with the map of public expenditure
  spending. The poverty map that is obtained can
  then be super-imposed on the map for any public
  spending

22
(No Transcript)
23
Ex-Post Evaluation of public programs (Ravallion
)

•  Randomization Only a random sample is allowed
  to participate to the program. Randomized out
  group is the counterfactual.
• Experiments may be either designed or natural
  Progresa vs. Bolsa Alimentacao
• Delayed participation of part of the population
  may be used to reach the same objective.
• But beware of anticipation bias
• Randomization ensures that treatment and control
  groups are alike along all dimensions relevant
  for program selection, observable and
  unobservable.
• Takes into account all partial and general
  equilibrium effects of program, as well as all
  behavioral responses. Ideal for measuring. Not so
  great at explaining.
•  

24
Ex-Post Evaluation of public programs (Ravallion
)

•   Matching When no randomization is available,
  must construct a comparison group. Objective is
  to approximate a control match participants to
  non-participants from a larger survey, on the
  basis of similarities in observed
  characteristics.
• The most common method is to match people on the
  basis of their ex-ante probability to participate
  to the program, these probabilities depending on
  their characteristics as well as those of the
  communities they live in (Propensity-score
  matching)
•  
• Draws on seminal work by Rosenbaum and Rubin
  (1983)

25
Ex-Post Evaluation of public programs (Ravallion
)

• Key problem with non-experimental data is that if
  any variables which affect selection into the
  program are not observed, they can not be
  included in X, and the approximation to the ideal
  counterfactual fails.
• If two waves of data are available in time (I.e.
  with a baseline survey and a follow-up survey),
  then at least the time-invariant unobserved
  variables may be netted out through double
  differencing
•  

26
Ex-Ante Evaluation of public programs
(Bourguignon Ferreira)

• Aims to simulate programs or program reforms
  which are not yet in existence. Complement to
  ex-post approach.
• In this approach, the treatment rather than the
  control is the counterfactual.
• The counterfactual incomes may be generated
  through
• Arithmetic micro-simulations (based on program
  rules)
• Behavioral micro-simulations (based on a model)

27
Ex-Ante Evaluation of public programs
(Bourguignon Ferreira)
28
Public Expenditure Tracking Surveys Dehn,
Reinnika and Svensson 2001

•   The need for special Public Expenditure
  Tracking Survey (PETS) comes primarily from the
  increasing evidence that budget allocations to
  social services (the basis for traditional
  benefice incidence analysis) are not consistent
  with the casual observation of what is really
  happening in the ground.  
• More evidence of government failures (corruption,
  leakages).
• Little known about transformation of budgets into
  services (the public sector production function)
• Household surveys show that quality of service
  important determinant of demand
•   PETS gathers information on flow of funds
  within the public sector from
• Participatory poverty assessments
• Service delivery surveys of households
• Public officials surveys

29

• Example Education sector in Uganda 1996
• Data from 250 schools and administrative units
• Only 13 percent of intended capitation grant
  actually reached schools (1991-95).
• Mass information campaign by Ministry of Finance
  (the press, posters)
• Follow-up surveys (PETS, provider surveys,
  integrity surveys, etc.)
• High leakage has also been found in other
  countries (Tanzania, Ghana, Zambia, Peru)

30
Part 2 Macroeconomic techniques, from robust to
more speculative.

• 1. Standard RHG approaches to macro-micro
  linkage
• "Micro-accounting"/RHG approach based on
  aggregate macro predictions (PovStat-SimSip-PAMS)
• The disaggregated SAM-CGE/RHG approach
  (Adelman-Robinson, Bourguignon and al. in the
  "Maquette, Loefgren, Robinson or Agenor and al.
  with IMMPA.)
• 2. Top-down "micro-simulation" approaches
  (micro-macro linkages)
• "Micro-accounting modules" linked to
  disaggregated macro models (Chen-Ravallion,
  McCulloch-Winters)
• "Micro-simulation modules" linked to
  disaggregated macro models (Bourguignon-Robilliard
  -Robinson, Ferreira-Leite-Pereira-Picchetti,
  Cogneau-Robilliard-van der Mensbrugghe)
• 3.Other issues for research and applications
• a) Fully integrated models (Townsend,
  Heckman, Browning-Hansen-Heckman)
• b) Accounting for general equilibrium effects
  of public expenditure programs
• c) Dynamic modeling and the proper treatment
  of growth

31
Evaluation of macro economic policies.Macro to
micro linkages
Macro framework, general/partial equilibrium

• A households character.

• p
• prices

• w wage

• L employt.

• R transfers

Instead of  exogenous and independent  shocks
like in Part 1, in Part 2 ? use  endogenous and
 dependent shocks to 'microsimulate' the effect
of policies on all individuals in the micro data
sets, and the poor ? some consistency
constraints will be  binding  (e.g., budget
envelope for social programs, real GDP growth,
etc.)
LAVs
Linkage Aggregate Variables
Household Survey (HHS), i individual households
32
Evaluation of macro economic policies. General
approaches and problems

• Before/after evaluation based on the
  observation of changes in standards of living Dy
  inputed to some policy change affecting
   jointly  (DL, DR, Dw, Dp, etc.)
• Problems Before/after evaluation techniques
  include other changes (DX) than policy (DL,
  DR,..) being evaluated ? difficulty to evaluate
  alternative policies by attributing changes to
  the effects of policy
• Counterfactuals Ideally possible to smulate
  changes in standard of living due to alternative
  macroeconomic policies, e.g., (DE, Dr) during a
  BoP crisis
• Problems Program design/implementation in
  crisis time - credibility?
• Top-to-Bottom approach Linking macro to micro
  data using Linkage Aggregate Variables (LAVs) to
  simulate macro-to-micro effects consistently
• Problems Weakest link (macro? micro?), garbage
  in, garbage out
• and

33
Evaluation of macro economic policies. General
approaches and problems
1995 Nobel Laureate in Economics Robert E. Lucas
Jr.
34
Standard RHG approaches to a macro-micro linkage
a) "Micro-accounting"/RHG and aggregate macro
predictions

• i. An elementary procedure
• Growth rate of output in sector k gk
• Growth rate of employment in sector k nk
• Effect on distribution (using a micro data
  base) given by
• Multiply income of all hhs in sector k (or RHG)
  by
• Reweigh all hhs in sector k (or RHG) by
• Evaluate new distribution, all poverty and
  inequality measures
• ii. More elaborated models
• Change arbitrarily distribution within sector k
• Change distribution endogeneously by
  distinguishing labor/non-labor income, so that gk
  is not uniform anymore (PAMS, Pereira da Silva
  and alii)
• iii. Main problems very much heterogeneity
  still missing likely strong selection behind
  nk

35

1. Standard RHG approaches to a macro-micro linkage
   b) The SAM-CGE/RHG approach

• i. Basic idea
• Aggregation properties allow separating the
  household population into groups. Only the
  aggregate behavior of these groups matters for
  the (general) equilibrium of the economy.
• Overall distribution of income or earnings
  studied under the assumption that the
  distribution of 'relative' income within
  Representative Household Groups is constant as
  given in a household survey - and also that
  their demographic weight is given.
• These approaches thus essentially focus on
  changes in the distribution between RHGs.
• ii. Working of standard (CGE) models (e.g.,
  Robinson)
• Full integration of RHGs' behavior within the
  model
• interaction of heterogeneous behavior in labor
  supply, consumption, savings, portfolio choice in
  the household sector with the production side and
  public policies through good and factor markets

36

1. Standard RHG approaches to a macro-micro linkage
   b) The SAM-CGE/RHG approach

• iii. Recent and current extensions
• Introduction of the monetary and financial
  sectors (IMMPA, Agenor and alii, Lewis
  Robinson)
• Limited by current theoretical knowledge of the
  working of financial markets.
• Introducing imperfect competition in different
  ways
• Economies of scale, economies of scope,
  oligopolistic behavior, bargaining on the labor
  market,
• Dynamics represented through a sequence of
  temporary equilibria linked by asset accumulation
  and demographics

37

1. Standard RHG approaches to the macro-micro
   linkage b) The SAM-CGE/RHG approach

• iii. Limitations
• Miss 'true' intertemporal behavior and important
  sources of growth ( public expenditures in
  particular)
• Constant within RHG distribution limitative in
  a dynamic framework
• All improvements over simple static Walrasian
  case make all the more acute the issue of
  empirically 'calibrating' the model and the
  confidence one may have on predictions
• The 'black box' risk
• iv. Final Remarks
• These techniques are 'simple', yet they are not
  widely used
• They capture only the 'between' (RHG) dimension
  of distributional changes, which empirically
  proves limitative
• They are ill adapted to the distributional
  aspects of growth

38
2. Top-down "micro-simulation" approach within a
macro-micro linkage approach
Macro model
Linkage AggregatedVariables (prices, wages,
employment levels)
Household income micro-simulation model
39
2. Top-down "micro-simulation" approach within a
macro-micro linkage approach
LAVs from above
Household income micro-simulation model

• Two distinct approaches to micro-module
• - "micro-accounting" no explicit change in
  behavior (envelope theorem argument), e.g.,
  Chen-Ravallion
• - "micro-simulation" change in behavior,
  possibly linked to (labor) market imperfections,
  e.g., Robillard, Bourguignon, Robinson and
  Ferreira, Leite, Pereira da Silva, Picchetti

40
2. Top-down "micro-simulation" approach a)
"Micro-accounting modules" linked to
disaggregated macro models

• i. Basic principles
• ?p, ?w, ?R obtained from macro model (CGE or
  other)
• observed in reference household
  survey
• Standard envelope theorem
• Where ?yi and stand for welfare income
  equivalent
• "Mobility" and distribution analysis can then be
  conducted on the set of ?yi and

41
ii. Example Evaluating the distributional
consequences of WTO accession for China

• Representing WTO Accession for China
• Reduce Chinas own protection to the lesser of
  the tariff binding or the 2001 applied rate
• Effect of trade reforms in China since 1995
  viewed as part of Chinas WTO accession process
  (counterfactual?)
• Separate impacts of tariff reductions to 2001 and
  the remaining reductions to 2007
• Elimination of textile clothing quotas for
  Chinas exports
• Removal of agricultural export subsidies for
  feedgrains (32) and plant-based fibers (10)
  (Huang and Rozelle, 2002).
• Liberalization of the service sectors (Francois,
  2002)

42
Example Incidence Curve from Chen and Ravallion
(China accession to WTO)
43

2. Top-down "micro-simulation" approach a)
"Micro-simulation modules" linked to
disaggregated macro models

• i. Micro-simulation model, basic idea
• Micro-simulation equivalent to introducing
  imperfect labor markets and occupation allocation
  models in previous framework. More behavorial
  content than micro-accounting
• Econometric model of household income is
  estimated allowing for full individual
  heterogeneity
• Income model (individual? households)
• Occupational choice (e.g., multi-logit)
• Simulates the effect on household income of
  modifying a subset of this model in accordance
  with predictions of the macro-model.

44

• ii. Link with macro model (CGE or other)
  counterfactual analysis
• Linkage aggregate variables (LAVs) given by macro
  model wages, prices, employment levels by
  status and labor segment
• Consistency 1 apply price changes as in
  accounting approach
• Consistency 2 Make occupational status
  consistent with macro employment levels by
  changing multi-logit intercepts
• Analogy with the operation of 'grossing up' a
  sample
• No feedback no explicit link with actual prices
  in macro model

45

• iii. Summing-Up layer structure macro-micro
  linkages approach
• From what precedes, proceeding top-down with
  three successive layers
• Aggregate model determining the standard macro
  aggregates (GDP, price level, exchange rate,
  interest rate), possibly in a dynamic way
• Disaggregated real CGE-type model, using the
  variables of the aggregate model as an input
• Micro-simulation module using output of previous
  models as linkage variables to make
  micro-simulation consistent with macro
  counterfactuals.

46
Recall Top-down "micro-simulation" approach
Objective reality test can approach replicate
real outcomes (HHS)?
General Equilibrium Macroeconomic Model CGE,
Macro-Econometric
Layer 1 Macro
Sectoral Disaggregation, Factor Markets ? Linkage
Aggregate Var For k representative groups of
households
Layer 2 Meso
Household Survey (HHS), i individual households,
Macro "consistent" changes in real household
incomes and change in the distribution of welfare
Layer 3 Micro
(yi) with poverty line, z, ? indicator of poverty
Pi for each household i and indicators of
within-group inequality (e.g., Gini, etc.)
47
2. Top-down "micro-simulation" approach vs.
standard CGE/RGH approach and actual outcomes

• iv. Comparing the top-down microsimulation
  approach with actual outcomes and the GCE/RHG
  approach what is more accurate?
• As a test, we compare counterfactual
  distributions obtained from the micro-macro model
  (Brazil) with actual outcomes from an existing
  HHS and then with the CGE/RH approach
• As a test, we compare counterfactual
  distributions obtained from the Indonesian CGE
  and the Brazilian micro-macro models
• a) Under the assumption that distribution of
  income within RHG (defined by the occupation of
  HH head) is constant
• b) With the top-down micro-simulation framework
  shown earlier.

48
Brazil Results Aggregate Poverty and Inequality
Indices (on aggregate, good results)
49
Example 1 Brazil, 1999 Financial Crisis, Results
of Simulationnominal changes in per capita
income after floating ER
50
Example 2RHG vs. Micro-simulation in the
Indonesian model
FULL (microsimulation) and RHG without and with
reranking

• Conclusions
• Aggregate results good, through complex LAV
  procedure
• counterfactuals are indeed different and
  macro-micro with microsimulation approach closer
  to actual outcome than RHG approach

51
2. Top-down "micro-simulation" approach
vii. Final Remarks introducing feedbacks
Macro model
Feedback, e.g., micro-transfers, minimum wage
Linkage AggregatedVariables (prices, wages,
employment levels)
Household income micro-simulation model
52
3. Other issues on the techniques.

• Fully integrated models
• Full heterogeneity of households estimated
  through panel data and interactions between them
  in labor and asset markets Heckman, Townsend,
  Browning, Hansen and Heckman. (Necessarily
  limited detail in structure of productive sector
  presently makes this approach unfit to the study
  of many development issues).
• b) Taking into account general equilibrium
  effects of public expenditure programs
• Spending on education, health or cash/in kind
  transfers to households has no direct productive
  effect in standard CGE or macro-econometric
  modeling.
• Possible to analyze distributional effect using
  microsimulation framework if some behavior is
  introduced demand for schooling or health
  services.
• But two difficulties arise i) most actual
  effects on distribution will be in the long-run
  (when kids will be adult) ii) initial policies
  likely to generate future general equilibrium
  effects at macro level (earning structure, growth
  rate) depending on the demand side of the economy
• We are presently not well equipped to handle
  these points

53
3. Other issues on the techniques.

• c) the issue of dynamic modeling and the proper
  treatment of growth
• Micro-simulation techniques largely remain
  comparisons of two cross-sections of households
  they describe what happens to a individual
  household which is itself representative of other
  actual households
• Dynamic modeling may involve simulating what
  happens to a given household after some policy
  has been decided at the macro level!
• Dynamic micro-simulation models used by
  demographers would permit to go in that
  direction. Also, integrated models, alluded to
  above, follow this kind of approach.
• These are not small undertakings. Yet it is
  necessary to continue research in that direction
  to be able to say something on the long run
  distributional effects of growth and macro
  volatility and some aspects of growth policies.

54
Final Practical considerations Usage of the
techniques to meet DCs policy challenges, if,
when, where and how

• Are these techniques for evaluation of public
  policies used? Some of these techniques are
  'simple', yet not all of them are widely used,
  why? Costs (training) and institutional
  implications for Ministries and agencies (Finance
  vs. Planning, political economy of budget
  process, etc). Aid agencies (IDA, DFID, AFD,
  etc) promoting evaluation, (e.g. F.
  Bourguignons DIME group)
• When these techniques are used, are they useful
  for policy-makers? They capture only the
  'between' (RHG) dimension of distributional
  changes, which empirically proves limitative.
  They are ill adapted to the distributional
  aspects of growth, but important in putting
  broader (poverty) perspective to decisions
• Where are these techniques for evaluation of
  public policies used? Examples below

55
Usage of these techniques in connection with
policy-making (with external assistance)

• Average Incidence Analysis (Tax Incidence
  Analysis and Public Expenditure Incidence
  Analysis)
• Most OECD countries. Also in many DCs
  particularly IDA Ghana (ISSER), Madagascar
  (INSTAT Cornell Univ.), Uganda (EPRC)
• Marginal Incidence Analysis
• Many OECD countries and India, using NSS 1994
  Indonesia, using SUSENAS 1981 1987 Vietnam
  using panel from VLSS 1993 1998
  Argentina,using public spending and census data,
  Ministry of labor team Brazil using PNADs
• Poverty mapping
• Many OECD countries Ecuador, Bolivia, Mexico,
  Panama, Nicaragua, Guatemala, South Africa,
  Madagascar, Kenya, Uganda, Malawi, Mozambique,
  Tanzania, Bulgaria, Albania, Thailand, Vietnam,
  Cambodia, Indonesia, China., Brazil, etc.
• Ex-post impact evaluation methods (randomization,
  PSM, double-dif)
• Many OECD countries. Also in many DCs,
  Argentina, Brazil, Kenya

56

• "Micro-accounting"/RHG approach based on
  aggregate macro predictions SimSIP/PAMS
• Latin America, Burkina-Faso, Thailand, Indonesia
• The disaggregated SAM-CGE/RHG approach
• IFPRI (US) country models, IMMPA-Cameroon,
  Brazil, many countries have GTAPLAVsHHS
• "Micro-accounting modules" linked to
  disaggregated macro models China, Colombia,
  Brazil, Many countries have GTAPLAVsHHS
• "Micro-simulation modules" linked to
  disaggregated macro models Indonesia, Brasil
• More sophisticated?
• Fully integrated models (Thailand, Madagascar)
  
• Accounting for general equilibrium effects of
  public expenditure programs (???)
• Dynamic modeling and the proper treatment of
  growth (???)

57
Usage of these techniques in connection with
policy-making, Brazil example with
Macro-stabilization Zero-Hunger, Bolsa-Familia
program

• Done on first principles Macro-stabilization
  necessary condition for poverty reduction
  (growth)
• Control inflation utmost importance for growth,
  poverty reduction
• Fiscal responsibility and Debt reduction are
  natural instruments
• Reduction of vulnerabilities (external)
• Done with Monitoring Evaluation Social
  programs are needed, limited resources
• Unification of several Federal programs
  (Bolsa-Escola, Vale-Gas, Bolsa-Alimentacao)? more
  eficiency (e.g., Oportunidades), but LOAS
  (old-age rural pension not done on PPP-basis by
  region and Social Security reform still a problem
• Emergence of CCT programs with incdentives and
  exit options
• Statistical apparatus available for good
  evaluation Cadastro Unico, PNADs and POF with
  Census and PIA

58
So..main messages?

• Micro-incidence analysis (average, behavioral) is
  easy to use, HHS are available, ex-ante and
  ex-post analysis can be conducted
• Many DCs using it
• Next steps inclusion in current policy
  frameworks
• Micro-macro linkages more costly in time,
  resources and skills (Indonesia, Brazil
  experiences entail cooperation of academia, IFIs,
  Government agencies)
• Important when most policies have macro content
• Difficult to maintain and implement (crisis-time
  is not a time for DIME, usually first-principles
  are used, e.g., Asian crises in 1998)

59
ENDCaminante no hay camino, se hace camino al
andar(Antonio Machado)