A Latent Class Callback Model for Survey Nonresponse

1
A Latent Class Call-back Model for Survey
Nonresponse

• Paul P. Biemer
• RTI International and UNC-CH
• Michael W. Link
• Centers for Disease Control and Prevention

2
Outline

• Motivation for the study
• Early cooperator effects (ECE) in the Behavior
  Risk Factor Surveillance System (BRFSS)
• Call-back models
• Manifest and latent
• Model extensions
• Application to the BRFSS
• Results
• Summary and conclusions

3
Terms and definitions

• Cooperators units that will eventually respond
  at some request or call-back
• Non-cooperators (also called hardcore
  nonrespondents) units that will not respond to
  any call-back

4
Terms and definitions (contd)

• Early cooperator Cooperators that respond at
  early calls (say, 5 or less)
• Later cooperators Cooperators that respond at
  later calls (say, 6 or more)
• Early cooperator effect (ECE) expected
  difference in estimates based on early vs. early
  later cooperators (say, )

5
Response rates as a function of number of call
attempts
Number of call attempts
6
Illustration 1- Have you ever been told by a
doctor, nurse or other health professional that
you had asthma?
Small ECE maximum of 5 calls is adequate
7
Illustration 2- During the past 12 months, have
you had a flu shot?
Larger ECE max of 5 call attempts may be
biasing Could consider other definitions of
early cooperator.
8
Why study ECE?

• Effort (and costs) could be saved if ECE is small
  
• If ECE is not small, adjustments may be applied
  to reduce it
• May need to adjust for HCNRs, not only later
  cooperators

9
What adjustments can be applied to reduce the ECE?

• Nonresponse adjustments
• Requires characteristics of nonrespondents
• Lack of information a limitation for some surveys
• Post-stratification adjustments
• Requires known target population totals within
  adjustment cells
• Variables limited to those available externally
• Call-back model adjustments
• Assumes response propensity is function of level
  of effort required to obtain a response and
  grouping variables
• Related work of Drew and Fuller (1980), Politz
  and Simmons (1949), others

10
ECE in the BRFSS

• Survey details
• One of the largest RDD surveys in the world
• Estimates the prevalence of risk behaviors and
  preventive health practices
• Monthly, state-based, cross-sectional survey
• Target population is adults in telephone hhs
• Data source 2004 survey with 300,000
  interviews

11
ECE in the BRFSS (contd)

• Early cooperator defined as responding with 5
  fewer call attempts
• Examined differences in
• demographic characteristics
• 10 selected health characteristics overall and by
  demographic domain
• ECE estimated by
• Data weighted by base weights only

12
Typical Values of ECE
13
Typical Values of ECE (contd)
14
Summary of the Results

• Early cooperators are different from later
  cooperators on many dimensions
• For most characteristics ECE is relatively small
• Less than 3 percentage points at aggregate level
• Rarely more than 3 points for domains
• For some characteristics, ECE may be important
• Other definitions of ECE also considered

15
Hardcore Nonresponse Bias

• Hardcore Nonrespondents Units that will not
  respond under the current survey protocol no
  matter the number of call-backs
• ECE does not include the bias due to hardcore
  nonrespondents
• Total nonresponse bias Bias due to cooperators
  who did not respond bias due to hardcore
  nonrespondents
• Adjusting for ECE may not remove bias due to HCNR

16
Call-back Models for Adjusting for ECE and HCNR
Bias

• General idea
• Estimate the response propensity for subgroups of
  the population
• Response propensity is modeled as a function
  level of effort (LOE) to obtain a response
• Two models are considered
• Manifest model (MM) Ignores HCNR
• Latent class model (LCM) Includes HCNR
• Includes a latent indicator variable to represent
  the HCNRs in the population
• Why latent?

17
Illustration for 5 Call-backs
1 interview 2 noninterview 3 noncontact
18
Illustration for 5 Call-backs
1 interview 2 noninterview 3 noncontact
19
Potential Advantages over Post-Stratification

• Post-stratification adjustments (PSAs) depend
  upon the availability of external benchmarks or
  auxiliary data
• Selection of control variables is quite limited
• Target populations also quite limited
• Adjust for ignorable nonresponse only

20
Potential Advantages over Post-Stratification

• Call-back model can rely only on internal
  variables
• Weighting classes can be defined for any
  variables collected in the survey
• Can be applied for any target population
• Greater ability to selected variables that are
  highly correlated with response propensity
• Adjust for ignorable and nonignorable
  nonresponse

21
Modeling Framework

• Simple random sampling
• Survey eligibility is known for all sample
  members
• No right censoring
• (i.e., all noncontacts received maximum LOE)
• Extensions to relax these assumptions are
  described in the paper

22
Incorporating the Model-based Weights

• Unadjusted estimator of the mean

Based on the sample distribution
Adjusted estimator of the mean
Estimated from the model
23
Two Models for Estimating

• MM (Manifest Model)
• Assumes all nonrespondents would eventually
  respond at some LOE (i.e., all nonrespondents
  have a positive probability of response)
• LCM (Latent class model)
• Incorporates 0 probability of response for the
  hardcore nonrespondents (HNCRs)

24
Technical Details
25
Notation
Levels of effort (LOE)
Outcome of LOE l where 1interview, 2
noninterview, 3noncontact
LOE associated with state S1 or 2
Grouping variable (weighting class variable)
26
Notation
Probability person in group g is interviewed at
LOE l
Probability person in group g is noninterviewed
at LOE l
Probability person in group g is never contacted
Number of sample persons in group g interviewed
at LOE l
Number of sample persons noninterviewed at LOE l
Number of sample persons never contacted after L
(max LOE) attempts
27
General Idea Outcome Patterns for 5 Call-backs

• Cooperator HCNR
• 11111 0
• 31111 0
• 33111 0
• 33311 0
• 33331 0
• 22222
• 32222
• 33222
• 33322
• 33332
• 33333

28
Likelihood for the Manifest Model

• This model is appropriate when
• Every sample member has a positive probability of
  responding at some LOE, or
• Adjustment for ECE only is desired

29
Likelihood for the Latent Class Model
Introduces a latent variable X where X 1, if
HCNR and X 2, if otherwise Appropriate when
some sample members have a 0 probability of
responding and adjustment for total nonresponse
(Later Cooperators HCNRs) is desired
30
Model Restrictions
The key assumptions regard the form of the
probabilities
, and
where
and
31
Model Restrictions (contd)
Manifest Model
(i)
, say, for l 3,...,L for Llt10
and (ii)
i.e., no HCNR,s and response probabilities are
equal for LOE 3,..., 9. The probabilities for
LOEs 1, 2 and 10 (if present in the model) were
not restricted
Latent Class Model

• is replaced by ...
• (i')

to represent the HCNR group in the model.
32
Results
33
Four Estimators were Considered

• Unadjusted estimator
• Estimator using MM estimates of
• Estimator using LCM estimates of
• Estimator using CPS estimates of
• i.e., usual PSA estimator
• treated as the gold standard

34
Comparison of the ECE for a Maximum Five
Callbacks Strategy Before and After MM Adjustment
35
Differences between PSA and Unadjusted and
Adjusted Estimates for a Maximum Five Callbacks
36
Estimating the Potential Bias Reduction

• BRFSS data do not exhibit very large nonresponse
  biases
• Therefore, consider a variable, Y, that has
  maximum nonresponse bias given the BRFSS
  nonresponse rates
• To do this, we form
• Yg BRFSS response rate for group g
• Compute the relative difference between
  unadjusted and adjusted estimates and the PSA
  estimate of the mean of Y

37
A Measure of Potential Bias Reduction

• Define the relative difference operator for any
  maximum LOE, L as

Gold standard estimator
38
Absolute Relative Differences (RDL) for
Unadjusted and Adjusted Estimators as a Function
of Number of Call-backs
39
Conclusions

• ECE for 5 call-backs is generally small, but can
  be moderately high for some characteristics
• The Manifest Model can be employed to reduce ECE
• The Latent Class Model can be employed to reduce
  total nonresponse bias (Later Cooperators HCNR
  bias)
• Future research should focus on
• Variable selection
• Comparisons of MSEs of the estimators
• Small/medium size sample properties
• Integration with other post-survey weight
  adjustments