Establishing Comparable Poverty Estimates in Serbia (and elsewhere

1
Establishing Comparable Poverty Estimates in
Serbia (and elsewhere)

• Jill Luoto
• January 25, 2007
• Western Balkans Poverty Analysis Course World
  Bank

2
Goals

• Introduce an adaptation of the poverty mapping
  methodology that enables the prediction of new
  poverty estimates that are strictly comparable
  when otherwise incomparable welfare estimates
  exist
• Present brief summary of findings for Serbia
• Lead everyone in an exercise using the PovMap
  software on Serbian data

3
The Problem

• Estimating the evolution of poverty in Serbia
  over recent years is complicated by a change in
  official surveys
• Living Standards Measurement Survey (LSMS)
  implemented in 2002 and 2003
• Household Budget Survey (HBS) implemented
  2003-2006
• The two survey instruments have different
  consumption modules. Some of the differences
  include
• LSMS included a list of item codes for
  consumption goods
• HBS utilized open diary format
• Different recall periods 1 week in LSMS, 2 weeks
  in HBS
• Different imputation procedures for housing rents
  and other expenditure items
• All in all, many differences in the way
  consumption and resulting poverty were estimated
  across surveys

4
Different Consumption Definitions Lead to

• Incomparable Poverty Estimates
• Lanjouw and Lanjouw (2001) offer real world
  examples where only slight changes in the
  definition of the consumption aggregate affect
  resulting poverty estimates dramatically
• For Serbia, the different consumption modules
  between LSMS and HBS have caused policymakers to
  generally consider their respective poverty
  estimates not to be comparable
• This leaves open the question as to what happened
  to poverty in Serbia between 2003 and 2005

5
Possible Solution

• Adaptation of the poverty-mapping methodology
  that aims to reconcile comparability of
  consumption definitions across surveys
• Other components of LSMS and HBS collect similar
  information
• Geographic information
• Household demographics
• Asset ownership
• Education and Labor Information
• Instead of imputing consumption definition from a
  survey into a census across space, impute from
  survey to survey across time
• Necessarily ensures an identical definition of
  consumption across data sources
• Implicit assumption that the relationship between
  consumption and its correlates remains stable
  over time

6
Methodology, In Brief

• Establish the completely comparable components
  between surveys
• Estimate a model of consumption in one survey
  using as explanatory variables only those
  correlates of consumption that are comparably
  defined across surveys
• Take the point estimates from that model of
  consumption and impute them into the other survey
  to estimate new consumption figures using same
  set of explanatory variables
• Derive new estimate of poverty using predicted
  consumption figures

7
ExampleTwo Surveys, Years 1 and 2
Find comparable survey components such that X1
and X2 have equal definitions, i.e.,
X1X2

• Examples of X variables
• Household Demographics
• Education of HH Members
• Asset Ownership
• Housing Quality Indicators

8
X1
Poverty Mapping typically imputes consumption
definition across space from a survey to a census
(within same year or short period of time)
X1
This adaptation imputes a consumption definition
from one survey into another from a different
year, i.e., across time
X1
X2
9
Implementation

• Gather all of the variables that collect similar
  information in LSMS and HBS (there are many)
• Generally 5 main categories for the types of
  information that are useful in describing a
  households welfare and commonly collected in
  surveys
• Geographic Information
• Demographics
• Education and Profession Variables
• Asset Ownership/Wealth Indicators
• Basic Health Information
• Define new variable in each dataset that has same
  definition (and name) across datasets
• Compare means, distributions of similar variables
  across surveys to ensure capturing same
  information

10
Finding common variables across surveys
HBS Questionnaire
LSMS Questionnaire
11
Restoring Comparability to Education Variables
My Definition
My variable definition matches exactly between
surveys
12
Importing data into PovMap

• We will be using subsets of pre-made Stata
  datasets from LSMS 2003 and HBS 2005 that have
  been matched and have identical variable names
• Go to File?New Project?Name your project
• Each dataset must have a hierarchical
  household-level identifying variable that can be
  truncated to identify the cluster
• Example HID32601 Cluster326

13
Stage 1 Checker Stage

• Compare distributions of variables across
  datasets
• If you think after this final stage of comparison
  that the variables are truly capturing the same
  information, set the variable to be included as
  a potential regressor
• Since were imputing from one survey to another
  from different year, its important to keep in
  mind that some variables are going to change over
  time, e.g., owning cell phones

14
Summary Statistics for Comparably Defined
Variables Geographic Variables
15
Summary Statistics Demographic Variables
16
You can also compare the entire distributions of
similar variables across data sources
17
Summary Statistics Education and Profession
Variables
18
Summary Statistics Housing Quality Indicators
19
Summary Statistics Durables Ownership Variables
20
Stage 2 Building a Consumption Model

• You've chosen all of the potential explanatory
  variables after all phases of screening
  (comparing surveys, comparing distributions) and
  now you move on to building your model of
  consumption
• Categorical variables are translated into a
  sequence of dummies
• Build models stepwise or intuitively choose
  explanatory variables using OLS
• Aim for highest R2 possible to best capture
  variation in household welfare levels
• Simultaneity and Omitted Variables Bias are not
  important for our purposes

21
Estimate consumption on subset of variables
comparably defined across surveys aim for
highest R2
Regression results from LSMS 2003
22
Stage 3 Cluster Effects

• Decompose the error term into a cluster effect
  and an idiosyncratic household effect
• This stage deals with modeling the cluster effect
• Since disturbance terms are likely to be
  correlated within clusters (due to unobserved
  geographic and other factors beyond those already
  included as regressors), this stage accounts for
  this by estimating a cluster random effect
• If you click on the "no locational effect"
  button, you take away this cluster effect from
  your estimation
• Underestimated standard errors

23
Stage 4 Idiosyncratic Model

• Here, still using the base survey data, you are
  trying to model the heteroskedasticity of the
  household idiosyncratic effect to allow it a more
  flexible form
• This stage tries to model the variance in the
  household-specific error terms as functions of
  the included X variables and combinations of
  variables
• Can use stepwise modeling or basic OLS or any
  other method to choose the explanatory variables
  that best explain variation in the household
  idiosyncratic effect
• Generally very low R2s in this stage (its all
  unobserved variation) 0.01-0.02 is sufficient

24
Stage 5 Household Effects

• Shows you a plot of the residuals from the model
  of the idiosyncratic household level error terms
• The Prediction Plot generally shows that your
  predictions arent so great here
• Empirical distribution of residuals can be
  compared to the normal and t-distributions (of
  varying degrees of freedom)

25
Stage 6 Simulation

• Here, we need to simulate the residual terms
  (both the cluster effect and the household
  idiosyncratic effect) since they are necessarily
  unknown in the latter survey (or census)
• Distributional forms You can either impose a
  normal distribution or allow for a more flexible
  semi-parametric distributional form using
  information from the predicted residuals from
  base data
• Choose the level of aggregation of your poverty
  estimates
• Choose poverty line, household size variable, and
  poverty indicators of your choosing
• Go!

26
Stage 7 Results

• Compare your resulting estimates of poverty with
  the baseline estimates from first survey

27
(No Transcript)
28
(No Transcript)
29
Conclusions

• For Serbia, this exercise suggests a gradual
  decline in poverty between 2003 and 2005
• Resulting poverty headcount estimate of 7.5
  based on models of consumption from both LSMS
  2002 and LSMS 2003
• Lower than official estimate of 9.1 for 2005
  based on consumption module of HBS
• Nearly 30 drop in poverty from LSMS 2003
  headcount estimate of 10.5 if results are
  believed
• This methodology can be used in a variety of
  settings to restore comparability of surveys to
  estimate evolution of poverty over time within a
  country or region
• Download the PovMap software at
  http//iresearch.worldbank.org/PovMap/index.htm