Microfinance and Home Improvement: Using Retrospective Panel Data to Measure Program Effects on Disc

1
Microfinance and Home Improvement Using
Retrospective Panel Data to Measure Program
Effects on Discrete Events

• Bruce Wydick
• Professor of Economics, University of San
  Francisco
• Visiting Professor , UC Santa Barbara
• joint with
• Craig McIntosh
• University of California at San Diego
• Gonzalo Villaran
• University of San Francisco

2

• Background
• Microfinance Summit As of January 2006, 3,133
  microcredit institutions have reported reaching
  113,261,390 clients, 81,949,036 of whom were
  among the poorest when they took their first
  loan.
• Still amazing that dont have robust results of
  positive microfinance impact (Armendáriz de
  Aghion and Morduch, 2005).
• Recent renewed emphasis on program impact
  appraisal (e.g. Easterly, 2006 Center for Global
  Development, 2006)

3

• An evaluation gap has emerged because
  governments, official donors, and other funders
  do not demand or produce enough impact
  evaluations and because those that are conducted
  are often methodologically flawed.
• --Center for Global Development (Saveduff,
  Levine, Birdsall, 2006 with E. Duflo, P. Gertler,
  etc.)

4

• Problems with lack of quality impact studies
• 1. Accurately measuring program impacts has
  historically been logistically difficult, time
  consuming, and costly.

5

• Problems with lack of quality impact studies
• 1. Accurately measuring program impacts has
  historically been logistically difficult, time
  consuming, and costly.
• 2. Many institutions would like to evaluate the
  effectiveness of their programs ex-post to
  implementation, creating problems with the
  establishment of baseline surveys, control
  groups, and other means of identification.

6

• 3. Use of instruments or program rules (e.g. Pitt
  and Khandker, 1998) to obtain program impacts is
  theoretically appealing, but practically
  problematic
• If available, instrumental variables will differ
  from one situation to the next.
• Finding instruments in a particular context
  strongly correlated with program access, but
  uncorrelated with impact variables, requires
  substantial ingenuity.
• Point complicates use of a standardized
  instrumental variable approach.

7

• 4. Matching Models -- creating artificial
  controls in order to identify treatment effects.
• (e.g. propensity scores, nearest neighbor, etc.)

8

• 4. Matching Models -- creating artificial
  controls in order to identify treatment effects.
• (e.g. propensity scores, nearest neighbor, etc.)
• Gomez and Santor (2003) use statistical matching
  model to identify the effect of group lending
  relative to individual lending among 1389
  individual and group borrowers among 1,389
  borrowers in Canadian lending institution.

9

• 4. Matching Models -- creating artificial
  controls in order to identify treatment effects.
• (e.g. propensity scores, nearest neighbor, etc.)
• Gomez and Santor (2003) use statistical matching
  model to identify the effect of group lending
  relative to individual lending among 1389
  individual and group borrowers among 1,389
  borrowers in Canadian lending institution.
• Problem Cannot control for unobservables.

10

• 5. Randomized experiments--become very popular as
  way of ascertaining impact of development
  programs.
• Maximum degree of exogeneity in treatment and
  control, allowing means of overcoming
  self-selection, endogeneity, and omitted variable
  bias (common to microfinance)
• Most elegant way of ascertaining impacts, and
  least controversial.

11

• Difficulties
• a) To create control group needed for
  identification of treatment, necessary that
  treatment withheld for some who desire it so
  impact can be measured on treatment group
  relative to the controloften undesirable or
  infeasible

12

• Difficulties
• a) To create control group needed for
  identification of treatment, necessary that
  treatment withheld for some who desire it so
  impact can be measured on treatment group
  relative to the controloften undesirable or
  infeasible
• b) Some treatments (e.g. microfinance) may take
  years to realize full effects on
  household--timeframe may not intersect with time
  one can hold off control group"bleeding" of
  control group.

13

• Difficulties
• a) To create control group needed for
  identification of treatment, necessary that
  treatment withheld for some who desire it so
  impact can be measured on treatment group
  relative to the controloften undesirable or
  infeasible
• b) Some treatments (e.g. microfinance) may take
  years to realize full effects on
  household--timeframe may not intersect with time
  one can hold off control group"bleeding" of
  control group.
• c) To avoid bleeding of control group, often
  short-term, but then only capture effects of
  initial adopters.

14

• d) Point estimates from randomized experiments
  are subject to influence of time-specific
  economic shocks occurring within the relatively
  narrow timeframe of the experimentunderstates
  true standard errors.
• e) Because randomized field experiments typically
  represent a snapshot of program impact over a
  short time frame, they are often unable to
  capture important dynamics of treatment impact.
• Ideally, we would like to understand how an
  intervention affects a treatment group over time.

15

• Our paper presents a methodology for ascertaining
  welfare changes brought about by development
  programs that may be applicable in a variety of
  contexts (explain later).
• Main Advantages
• Uses a single wave of cross-sectional surveying.
• Impact evaluation can be undertaken ex-post.
• No firm requirement for standard control group.
• Allows for a dynamic analysis of impacts

16

• Our methodology appropriate when
• Program has existed for a number of years.
• Has been phased in over time in different
  geographical regions or identifiably separate
  populations for reasons that are independent of
  dependent variables.
• Stable populations with little geographical
  movement.

17

• Methodology uses a single cross-sectional survey
  to create a retrospective panel data set based on
  discrete, memorable events in the history of
  households.
• e.g. install indoor plumbing, new house, purchase
  of first cell phone, miscarriages, infant deaths,
  land purchases etc.
• Identification of impact rests in analyzing the
  timing of these events with respect to the timing
  of treatment.
• Test is for differences in the probability of
  these major events within window surrounding the
  treatment.
• (Post vs. Pre--under conditions of exogeneity)

18

• We apply methodology to studying the effects of a
  microfinance program in rural Guatemala on home
  improvements

19

• We apply methodology to studying the effects of a
  microfinance program in rural Guatemala on home
  improvements
• Study discrete changes in the probability of
  major dwelling improvements, upgrades of walls,
  roofs, floors, the installation of indoor
  toilets, and the purchase of new land
  compilation of these.

20

• We apply methodology to studying the effects of a
  microfinance program in rural Guatemala on home
  improvements
• Study discrete changes in the probability of
  major dwelling improvements, upgrades of walls,
  roofs, floors, the installation of indoor
  toilets, and the purchase of new land
  compilation of these.
• Use linear probability estimator that
  incorporates household and year fixed-effects.
  (Chamberlin, 1980)

21

• Sneak Preview of Results
• Microfinance loans for enterprise expansion is
  likely to exhibit significant, positive effects
  on some dwelling upgrades, especially to walls
  floors.
• Roofs uncertain.
• Apparently not for toilets and land.

22

• Identification of impacts is achieved through the
  existence of counterfactual, i.e. what would have
  happened to treatment group in the absence of a
  particular treatment.
• Counterfactual in a randomized field experiment
  in microfinance is the resulting level or change
  in impact variables realized within a subset of
  borrowers in the control group who desired credit
  but were prevented by researchers from receiving
  it.

23

• Counterfactual that yields identification in our
  methodology is difference in probability of
  discrete events among those who received the
  treatment in separate years, controlling for
  these differences in years with village- and
  year-level fixed-effects.

24

• Main contributions
• 1. Offer a sequence of steps that include
  diagnostics on the data to check for supply-side
  demand-side endogeneities in the rollout of a
  program.
• 2. Establish framework for thinking about when
  and how retrospective panel data can be used in
  impact analysis.
• 3. When data meets certain diagnostic criteria,
  allows us to examine the dynamics over
  probability of these discrete events before and
  after treatment test for significance of a type
  of treatment effect.

25

• Steps involved in methodology
• Part A Survey

26

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different populations.

27

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different
  populations.
• Step S2 Carry out random survey of program
  participants who have been given access to the
  treatment in different time periods.

28

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different
  populations.
• Step S2 Carry out random survey of program
  participants who have been given access to the
  treatment in different time periods.
• Step S3 Identify discrete historical changes
  with a theoretical basis for causality from the
  treatment create historical panel.

29

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different
  populations.
• Step S2 Carry out random survey of program
  participants who have been given access to the
  treatment in different time periods.
• Step S3 Identify discrete historical changes
  with a theoretical basis for causality from the
  treatment create historical panel.
• E.g. fresh water ? reduced infant mortality

30

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different
  populations.
• Step S2 Carry out random survey of program
  participants who have been given access to the
  treatment in different time periods.
• Step S3 Identify discrete historical changes
  with a theoretical basis for causality from the
  treatment create historical panel.
• E.g. fresh water ? reduced infant mortality
• E.g. smallpox vaccine ? lower instances of
  smallpox

31

• Steps involved in methodology
• Part A Survey
• Step S1 Identify a program that has been phased
  in over a number of years in different
  geographical areas or among different
  populations.
• Step S2 Carry out random survey of program
  participants who have been given access to the
  treatment in different time periods.
• Step S3 Identify discrete historical changes
  with a theoretical basis for causality from the
  treatment create historical panel.
• E.g. fresh water ? reduced infant mortality
• E.g. smallpox vaccine ? lower instances of
  smallpox
• E.g. microcredit access ? higher enterprise
  profits ? more rapid home improvements

32

• Steps involved in methodology
• Part B Econometrics

33

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.

34

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect

35

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect
• Step E3 Testing for Demand-Side Endogeneity

36

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect
• Step E3 Testing for Demand-Side Endogeneity
• Step E4 Estimation of the Take-up Effect

37

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect
• Step E3 Testing for Demand-Side Endogeneity
• Step E4 Estimation of the Take-up Effect
• Step E5 Treatment Window Regression and F-test
  of Take-up Effects

38

• 2005 (BASIS/USAID funded) survey of 218
  households located in 14 different villages near
  Quetzaltenango, Guatemala.
• MFI Fe y Alegria 3,000 new clients/year
• Questionnaire ascertained changes in different
  categories of dwelling improvement upgrades to
  walls, roofs, floors, plumbing, and increases in
  land.
• Each borrower was asked about changes in these
  variables during the history of the household,
  and the timing of these changes.

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52

• Empirical Steps
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Diagnostics
• 1A Is there endogeneity in the levels of the
  pre-treatment outcome?
• (Regress average pre-treatment outcome on the
  year in which credit was offered to the village.)
  

53

• Empirical Steps
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Diagnostics
• 1A Is there endogeneity in the levels of the
  pre-treatment outcome?
• (Regress average pre-treatment outcome on the
  year in which credit was offered to the village.)
  
• 1B Is there endogeneity in the pre-treatment
  trend? (Regress average of the 1st difference of
  the pre-treatment outcome on year credit
  offered.)

54

• Empirical Steps
• Step E1 Check for supply-side endogeneity in
  the rollout of a program.
• Diagnostics
• 1A Is there endogeneity in the levels of the
  pre-treatment outcome?
• (Regress average pre-treatment outcome on the
  year in which credit was offered to the village.)
  
• 1B Is there endogeneity in the pre-treatment
  trend? (Regress average of the 1st difference of
  the pre-treatment outcome on year credit
  offered.)
• 1C. Is the rollout endogenous to shocks?
• (Run fixed effects using only pre-treatment data
  with dummy for 1st lead of year credit offered.)

55
(No Transcript)
56

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program. ?
• Step E2 Estimation of a Retrospective Intention
  to Treat Effect

57
(No Transcript)
58
(No Transcript)
59

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program. ?
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect ?
• Step E3 Testing for Demand-Side Endogeneity

60

• We estimate the following equation
• (1)
• where yit is a bivariate dependent variable that
  is equal to 1 if household i has housing upgrade
  in year t.
• vj is a village-level fixed effect,
• ?t is a year-level fixed effect, Xn are controls
• uit is an error term, and treatment dummy
  variable T is equal to 1 if household i first
  received a microfinance loan (or began receiving
  remittances) periods ago, and zero
  otherwise.

61
Data Set Example
62

• ...which can be adjusted for
• demand-side endogeneity
• (2)
• where yit is a bivariate dependent variable that
  is equal to 1 if household i has housing upgrade
  in year t.
• vj is a village-level fixed effect,
• ?t is a year-level fixed effect, Xn are controls
• uit is an error term, and treatment dummy
  variable T is equal to 1 if household i first
  received a microfinance loan (or began receiving
  remittances) periods ago, and zero
  otherwise.

pre-treatment interactive term
63
Reason Demand-side Endogeneity Suppose
borrowing is an endogenous decision because
people borrow in good economic times ? creates
upward bias, ds gt 0 Suppose borrowing is an
endogenous decision because people borrow when
they are in difficult economic times ? creates
downward bias, ds lt 0
64
(No Transcript)
65

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program. ?
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect ?
• Step E3 Testing for Demand-Side Endogeneity ?
• Step E4 Estimation of the Take-up Effect

66
(No Transcript)
67
(No Transcript)
68

• Steps involved in methodology
• Part B Econometrics
• Step E1 Check for supply-side endogeneity in
  the rollout of a program. ?
• Step E2 Estimation of the Retrospective
  Intention to Treat Effect ?
• Step E3 Testing for Demand-Side Endogeneity ?
• Step E4 Estimation of the Take-up Effect ?
• Step E5 Treatment Window Regression and F-test
  of Take-up Effects

69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77

• Conclusions
• Presented a methodology for ascertaining the
  impact of development programs such as
  microfinance that offers several advantages
• Can be used within existing client base.

78

• Conclusions
• Presented a methodology for ascertaining the
  impact of development programs such as
  microfinance that offers several advantages
• Can be used within existing client base.
• Data can be collected in single x-sectional
  survey

79

• Conclusions
• Presented a methodology for ascertaining the
  impact of development programs such as
  microfinance that offers several advantages
• Can be used within existing client base.
• Data can be collected in single x-sectional
  survey
• Illustrates timing and dynamics of impact

80

• Conclusions
• Presented a methodology for ascertaining the
  impact of development programs such as
  microfinance that offers several advantages
• Can be used within existing client base.
• Data can be collected in single x-sectional
  survey
• Illustrates timing and dynamics of impact
• Other (easier) applications Fresh water systems,
  Nutrition programs, Cash transfers, Vaccinations
  Electrification