The Good Shepherd

1
The Good Shepherd

• Sample Design and Control for
• Wealth Measurement in the
• Survey of Consumer Finances
• Arthur B. Kennickell
• Federal Reserve Board

2
Thanks!

• Martin Frankel Stephen Heeringa Roderick
  Little Donald Rubin Fritz Scheuren Louise
  Woodburn
• FRB Brian Bucks Gerhard Fries Kevin Moore
  Michael Neal Brian Presolpsky
• SOI Barry Johnson
• NORC Leslie Athey SCF field staff
• SCF respondents
• Opinions are those of the author alone and not
  necessarily those of the FRB.

3
Sampling

• Surveys are measurement engines
• Sample is the statistical foundation
• (Neyman 1934 essential reading)
• Tool for guiding operations
• Key to post-survey adjustments
• (Ignoring interview content/administration)

4
Frequentist/Model-Based Sampling

• Replicability and randomness are key
• Bias means systematic error that persists across
  replications of a sampling process
• NOT that we get the right answer every time
• Efficiency characterizes how far from true value
  a given sample est. might tend to be

5
Sampling Error for Mean(y)
6
1 Y1000 99 Y1 Error
7
1 Y1000 99 Y1 Error
8
Limit potential variability

• Stratification
• Finding an adequate basis for stratification
• Particularly important with wealth data
• SOI data for the SCF area-probability sample

9
Frame Problems

• Income not wealth (whatever that is!)
• Idiosyncratic rates of return
• Legal and illegal avoidance
• Taxpayers not households
• Changes in household structure
• Use models to translate into an appropriate index

10
SCF Wealth Index Model
11
Why Index Might be Inefficient

• Inappropriate rates of return
• Bad model estimates
• Income fluctuations
• Changes in labor force participation
• Movements in financial markets
• Changes in tax laws
• Convenience

12
Variation in income over time
13
Net worth by WINDEXM2001
14
Sample Not what one gets

• Locating/contacting issues
• Refusals
• Differential application of effort

15
Sample Management
16
Effects of Systematic Management

• Break endogeneity between effort and nonresponse
• Ability to estimate nonreseponse models
  reflecting respondent behavior
• More ethical treatment of respondents equal
  opportunity

17
Post-Survey Adjustments

• Use sample/external information
• Helpful if sample identifies key dimensions of
  nonresponse

18
The Future

• Given data (easy!) straightforward to
  design/improve unbiased/efficient sample
• Nonresponse is likely to be a permanent problem
• Administrative (register) data may help to some
  degree but frameworks differ
• Additional work needed on management of sample
  and nonresponse