Qualitative and Quantitative Poverty Appraisal: Maximizing Complementarities, Minimizing Tradeoffs

1
Qualitative and Quantitative Poverty
AppraisalMaximizing Complementarities,
Minimizing Tradeoffs

• Chris Barrett
• Cornell University
• Presentation at Wageningen, August 26, 2002

2
This seminar draws on presentations and
discussions at March 2001 conference at Cornell
University, summarized in Ravi Kanbur, ed.,
Q-Squared Combining Qualitative and Quantitative
Methods In Poverty Appraisal (Delhi Permanent
Black, forthcoming), as well as on the insights
of colleagues in several multidisciplinary
collaborative research projects.
3
Significant recent progress in both qualitative
(QUAL) and quantitative (QUANT) methods - rapid
rise of PA methods - emergence of widespread,
nationally representative household
survey data.
4
Are QUAL and QUANT complements or
substitutes?There is considerable conflict
among the practitioners of each does that mean
the methods necessarily conflict
too???Importance of being self-reflexive and
self-critical
5
Dimensions of QUAL-QUANT Difference Be clear
about focus of question(1) Data collection
methods(2) Data types (3) Data analysis
methods(4) Audience
6
Qual-Quant DimensionsData collection methods
General Specific
Census
SR Surveys
PRA
Autobiography
Passive Active Population Involvement in
Research
7
Qual-Quant Dimensions

• Qualitative Quantitative
• Data types
• Categorical Ordinal Cardinal
• Each data collection method can yield both
  non-numerical and numerical data

8
Qual-Quant Dimensions

• Qualitative Quantitative
• Data analysis methods
• Inductive Deductive
• Related to the specific-general data collection
  methods distinction, theres often (not always) a
  difference in analysis methods.

9
Qual-Quant Dimensions

• Audience
• Local community Global/national
  policymakers
• QUAL researchers often worry out loud about local
  empowerment and the intrinsic value of the
  research process. QUANT types tend to worry
  about big picturetake home messages
  speaking truth to power

10
Key advantages of QUAL approaches

• Allow more immediate probing in response to
  unanticipated results (adaptability)
• More nuanced and textured for complex,
  unmeasurable concepts (e.g., power, opportunity,
  security)
• Let subjects speak for themselves

11
Key advantages of QUANT approaches

• Use of sampling frames and randomization reduces
  inferential bias coincidence and causality
• Uniformity/structure in design/definitions
  fosters replicability over time (longitudinal
  analysis) and across samples (comparative
  analysis)
• Easily aggregability few scaling up problems

12
Myths about QUAL-QUANT differences

• One more/less extractive than the other
  (ethical superiority)
• One more/less contextual than the other
  (historical superiority)
• One inherently numerical/non-numerical
• (statistical superiority)
• (4) One more rigorous than the other
• (scientific superiority)
• Bad practice is bad practice, whatever the
  method..
• Key question when and how is good practice
  within one strand still wanting? How can the
  other fill the blanks?

13
Mixing methods

• When brought together, QUAL and QUANT rarely have
  similar status, especially in policy discourse,
  where aggregability and the illusion of
  precision commonly dominate.
• Improve analysis by mixing the two taking the
  con out of econometrics, generalizing beyond
  the part of participatory methods

14
Why mix methods?

• QUAL can improve QUANT by
• Improving survey/instrument design. Data are
  social products, so need to understand source
• Improving specification of formal models
• Improving statistical inference
• Identifying suitable instrumental variables,
  exclusionary restrictions, etc.
• Shedding light on outliers (It helps to have had
  tea with an outlier Biju Rao)
• Highlighting likely sources of measurement error
  (the Chai stall error Ron Herring)

15
Why mix methods?

• QUANT can improve QUAL by
• Reducing researcher-induced bias by structure and
  replicability
• Facilitating comparability
• Facilitating aggregability
• Broadening the audience for results
• Fostering more precise criteria for demonstrating
  causal relationships

16
Different methods of mixing

• Sequential mixing or classical integration
• Practitioners of each method do their best with
  their own tools on same problem, then triangulate
• Examples Fields work on SA labor markets
• Shafers work on intrahh inequality in
  west Africa
• BASIS project on natural capital and
  dynamic poverty traps

17
Different methods of mixing

• Simultaneous mixing or Bayesian integration
• Iterative approach to using one method to inform
  another, then back to the first, etc., keeping
  multiple methods interactive throughout the
  research process.
• Feedback loop yields a homeostatic research
  mechanism
• ethnography precedes participatory which in
  turn precedes survey in dictionary and should
  in field, too!
• Ongoing, creative tension between methods helps
  ensure originality, robustness and relevance of
  results

18
Different methods of mixing

• Example Pastoral Risk Management (PARIMA)
  project based on multidisiplinary integration
• (a) What does it mean to poor or vulnerable in
  this setting? How does this vary across
  individuals, households, communities and time?
  asking the right questions or the right people
  at right time?
• (b) Derivative from (a), are we measuring the
  correct variables and in the right manner?
• (c) Is our inference consistent (i) across
  methods (a test of robustness) and (i) with local
  understandings of the problem(s) (a test of
  relevance)?

19

• Tools developed/employed
• - Participatory risk mapping (Smith et al. WD
  2000, JDS 2001) to identify relevant threats
  (e.g., human health) open-ended,
  spatially-explicit, pseudo-cardinal
• - Quarterly repeated surveys with open-ended
  sections and mixed modules
• (i) complex property rights climate
  forecasting, resource conflict land use history
  livelihoods strategies, etc.
• (ii) complementarity at multiples levels of
  analysis and different methods (e.g., livestock
  marketing with data from households, markets and
  traders)

20
Example Participatory risk maps of rainfall and
drought risk (Smith et al. 2001 JDS)
21
Walking On Both Legs

• Development scholars and practitioners
  increasingly recognize the complementarity
  between qualitative and quantitative methods
• There are big gains to be enjoyed from
  relatively small movements along the QUAL-QUANT
  axes in any of several dimensions.
• Tradeoffs grow, however, so multidisciplinary
  mixing seems best, whether sequential or
  simultaneous, to take advantage of inherent
  complementarities from diversity of methods.

22
Walking On Both Legs

• But much remains to be done
• Need work on
• (i) vocabulary
• (ii) field methods
• (iii) data cross-referencing
• (iv) fostering respectful dialogue