Disability questions in censuses and surveys Methodological issues: proxy response, nonresponse and

1
Disability questions in censuses and surveys
Methodological issues proxy response,
non-response and mode of administration
Fifth Meeting of the Washington Group

• 21-23 September 2005
• Montserrat López-Cobo

2
Proxy response

• Are proxy-responses different from
  self-responses? Why?
• Do all proxies provide the same quality
  responses? Who is the best proxy?
• Is there an effect on the quality of data? Can we
  measure it? Can we correct it?

3
Proxy responseLevel differences 1

• It is generally accepted that proxy- and self-
  responses differ
• Many studies found that proxy-respondents
  underreport disability-related information
• Disability
• Impairments of emotion and Pain
• Limitation of activities, long-term disability,
  need for assistance with ADL
• Chronic condition, days of activity restriction,
  bed disability days
• General health events and conditions

4
Proxy responseLevel differences 2

• ... While some other studies found that proxies
  overreport
• Cognitive status, Mild Cognitive Impairment
• ADL disability
• Functional health, when the self-respondent is
  demented

5
Proxy responseUnderstanding self-proxy
differences 1

• Differences in cognitive procedures (1/2)
• Type and level of available information
• Proxies report impairments or conditions that are
  more observable or mentioned to them
• Proxy reports are more accurate for conditions
  that are serious, painful, persistent or
  potentially life threatening
• If person under 65 proxies underreport
• If person 65 and over proxies overreport
• Proxy information is more stable. Self
  information is more dynamic
• Self-responses are less consistent across time

6
Proxy responseUnderstanding self-proxy
differences 2

• Differences in cognitive procedures (2/2)
• Proxies engage in less extensive cognitive
  process. Provide heuristic-based responses
• Proxies rely more on inferences and estimations.
• Proxies overreport disabilities aparently related
  to a previously reported disability and
  underreport disabilities seemingly unrelated to
  the previously reported.
• Proxies are less affected by social desirability
  concerns
• Proxies are more willing to report disability

7
Proxy responseThe best proxy-respondent

• Social relationship with the selected person
• The closest the relationship, the most accurate
  the proxy-response
• Spouses responses are better than other proxies
• Proffesionals and caregivers provide more
  accurate information than lay proxies
• Length of relationship
• Positive effect in convergence self-proxy
• Memory capacity of respondent
• Affect to data quality

8
Proxy responseMeasuring and correcting bias

• Types of methods
• 1) Traditional
• First step Measure the bias by comparing proxy-
  estimates with self- estimates
• Second step Introduce statistical control
  adjusting for demographic or health-related
  variables
• 2) New strategy
• First Set assumptions about the nature of
  differences between self- and proxy-responses
• Second Model and estimate these differences
  based on the assumptions

9
Mode of administration
10
Mode of administrationSources of differences
among modes 1

• Differences in the sample
• Population coverage
• Response rates
• Differences due to social context within which
  data are collected
• Contact with interviewer
• Non-verbal cues
• Social desirability bias Reduced with
  self-administered questionnaires

11
Mode of administrationSources of differences
among modes 2

• Differences intrinsic to the mode
• Visual materials cannot be used by phone
• Complexity of the questionnaire is limited by the
  mode
• Stimuli in a visual mode ? Primacy effects
  Stimuli in a hearing mode ? Recency effects
• Differences in responses involving long lists
• Order effects
• Effects of the mode on the interviewer

12
Mode of administrationResults from comparative
studies 1

• Telephone interviews
• Do not underrepresent people with disabilities
• Sometimes is not as well accepted by population
  as face-to-face interviews
• Proved useful to assess mental health using
  recommended instruments (GHQ-12, CIS-R, CIDIS),
  the Expanded Disability Status Sacle
• Telephone vs Mail
• Mail responses report poorer health and more
  chronic conditions than Telephone
• Differential non-response rates by age
• Elder NR (Telephone) gt NR (Mail)
• Young NR (Telephone) lt NR (Mail)
• General Item NR (Telephone) lt Item NR (Mail)

13
Mode of administrationResults from comparative
studies 2

• Face-to-Face vs Mail
• Health differences between respondents and
  non-respondents by mode non-response bias
• Non-respondents and late respondents to Mail are
  more cognitively impaired and more disabled than
  respondents (among elderly).
• Non-respondents to F-t-F are similar to
  respondents.

14
Mode of administrationResults from comparative
studies 3

• Telephone/CATI vs Face to Face
• Differential non-response rates NR
  (CATI) gt NR (F-t-F)
• Measurement bias MBias (CATI) lt MBias
  (F-t-F)
• No differences between CATI and F-t-F for reports
  on chronic conditions, activity limitations and
  disability rates
• CATI vs CAPI
• Differential non-response rates by age
  
• Elder NR (CATI) gt NR (CAPI)
• Young NR (CATI) lt NR (CAPI)

15
Mode of administrationResults from comparative
studies 4

• CASI
• Reduces social desirability bias
• Enhances the feeling of privacy
• Respondents generally like CASI
• Technological possibilities have a positive
  influence on data quality (minimizing errors)
• Respondents self-disclosure is higher in CASI

16
Mode of administrationAn experience interviewing
disabled people by telephone

• Disabilities Physical, sensory, mental illness
  and mental retardation
• Challenges addressed communication, fatigue and
  cognitive issues
• Questionnaire design
• Eliminate soft consonant sounds (s, z, t, f and
  g) to overcome high-frequency hearing loss
• Build in "breaks" for respondents to let them
  rest
• Incorporate neutral encouragement to avoid
  drop-outs
• Design checks for unexpected responses
• Use structured probes for questions that might be
  difficult to understand

17
Mode of administrationAn experience interviewing
disabled people by telephone

• Interviewer training and supervision
• Usual background and purpose of study
• Training on challenges likely to face
• Sensitive exercise regarding the treatment to the
  disabled person
• Guidance to overcome each of the challenges
• Support to interviewers and reduction of stress
• Other recommendations
• Interviews take longer ? Multiple sessions may be
  required

18
Non-response
19
Non-response and Disability

• Components of non-response
• Non-contact
• Non-cooperation
• Non-contact the household pattern of disabled
• Are older and likely to live in non-metropolitan
  areas ? higher probability of being contacted
• Less likely to live with children ? lower prob.
  of contact
• May be more fearful of opening doors to strangers
  ? lower prob. of contact

20
Non-response and Disability

• Non-cooperation exchange theory vs social
  isolation theory
• Exchange theory Persons who feel that the survey
  sponsor has provided (or could provide) benefits
  to them are more likely to cooperate
• Persons with disabilities might be expected to be
  more cooperative with a government-sponsored
  survey
• Social isolation theory People who are isolated
  from the mainstream society feel less
  responsibility toward government and are less
  likely to cooperate
• Persons with disabilities might be expected to be
  less cooperative

21
Non-response and DisabilityResults from studies 1

• Persons with severe disability are more likely
  than persons with less severe disability to be
  contacted and to cooperate, but when they are
  interviewed they are less likely to answer for
  themselves instead, proxy and assistant
  respondents tend to answer for them.
• Elder people with disabilities respond sooner
  than young and non-disabled to mail surveys.
• These findings support Exchange theory
• Self-perceived memory problems increase item
  non-response and I dont know answers.
• Not preceived memory deficits implies inaccurate
  information on items requiring recall.

22
Non-response and DisabilityResults from studies 2

• Health of non-respondents is worse than
  respondents in terms of stroke, Basic ADL,
  mobility disabilities, self-rated health and
  mortality rates.
• Respondents with poorer physical functioning
  and/or limiting long-term illness have higher
  non-response rates.
• These findings support Social isolation theory

23
Recommendations for including disabled people in
interview surveys
24

• Bias due to exclusion is amplified in surveys
  where disability is a key measure of interest
• Recommendations
• Include institutionalised population in samples
• Carry out the interview in a private environment
• Provide adaptative technologies and procedures
  (of primary importance in visual, hearing and
  speech impairment).
• Use simple and clear questions

25

• Recommendations
• Special training for interviewers
• Provide alternative modes of administration
• If proxy is used
• Respondents assessments of proxys answer is
  valuable
• The proxy should be nominated by the selected
  respondent

26
Final conclusion

• Proxy response validity depends on factors such
  as proxy choice and distinction, topic
  investigated, health condition of the selected
  person...
• Mode of administration (unique or a mix-mode)
  should be decided taking into account topic
  investigated, population objective of the survey,
  expected acceptability of the mode by the
  population, technological possibilities...

27
Final conclusion

• Different patterns of response might be
  indicative of bias which can affect estimates.
• Non-response is one of the sources of data error.
  But not the unique... Emphasis should be given
  not only to minimazing non-response rates but
  also to estimation and control of measurement
  error.

28
Fifth Meeting of the Washington Group

• 21-23 September 2005