Unit non response in facetoface and web surveys' Some comparisons'

1
Unit non response in face-to-face and web
surveys. Some comparisons.

• Geert Loosveldt
• Dirk Heerwegh
• Centre for Survey Methodology
• Department of Sociology
• K.U.Leuven

2
1. Introduction

• Unit non response in survey methodology
• what are the causes and consequences?
• Not at random---gt biased estimates
• evaluation of non response
• comparison of the survey data with benchmark
  information
• panel data

3

• Panel data
• data from previous waves
• no evaluation of the first wave
• hypothesis about unit non response
• generalisation of results
• research question
• Are explanations of unit non response in
  face-to-face panels useful to explain unit non
  response in web panels ?

4
2. A limited model for face-to-face panel
refusals

• Some relevant factors (e.g. social context,
  survey design factors) are not in the model
• survey experience
• insufficient cognitive abilities --gt difficulties
• low level of interest--gt difficulties
• difficulties --gt unpleasant experience --gt
  decision to participate next time
• background characteristics have indirect effects
• effect on interest and difficulties

5
(No Transcript)
6

• Data
• panel data Flemish part of Belgian General
  Election
• first wave in 1991 response rate 62.7
• second wave in 1995 response rate 68.3
• Test of the model
• the probability to participate in the second wave
  is related to
• the number of dont know answers
• the number of inconsistent answers
• the interviewers evaluation of the respondents
  ability

7

• Higher levels of political interest tended to be
  associated with a higher probability to
  participate in the second wave
• conclusion
• ideas of the limited model are confirmed
• the more difficult the task the less likely to do
  it again
• the more negative the cost benefit analysis after
  the first interview the less likely to
  participate in the second interview
• question other relevant variables in the cost
  benefit analysis

8

• The effect of utilitarian individualism
• The benefits for respondents are small
• the cost benefit analysis will be different for
  respondents focused on individual gains
• utilitarian individualism involves striving for
  personal interests and success
• Respondents of the second wave have a
  significantly lower degree of utilitarian
  individualism than refusals
• importance of cost benefit analysis

9
3. Modelling panel attrition in web surveys

• Main hypothesis
• web survey panel attrition is affected by the
  same factors and in a similar way as in a face
  -to-face panel
• specifications
• less difficulties in filling in the first wave of
  the web survey and a higher level of interest in
  the topic are associated with a higher
  probability of participating in the second wave
• respondents with a higher level of utilitarian
  individualism will have a greater tendency to
  refuse

10

• Data for the web survey
• 3094 sample units from a publicly available
  database on the Internet
• first wave
• September 2002
• topics Euro, attitude towards immigrants,
  Internet usage,.
• login rate 24.6 response rate 22.6
• question asking for cooperation in the future
  82.8 (581 respondents) yes answers

11

• Second wave
• January 2003
• only respondents with a positive answer
• topics attitudes and behavior in road traffic,
  utilitarian individualism, ...
• Login rate 69.8 response rate 65.6
• Third wave
• november 2003
• topics attitudes towards refugees, relative
  deprivation,
• login rate 55.9 response rate 53.9

12

• Results
• Respondents difficulties
• indicator 1 incorrect answers to three knowledge
  questions about Muslims

13

• Indicator 2 item non response to five sets of
  opinion questions
• indicator 3 respondents evaluation of the
  difficulty of the survey
• associated with the intention to participate
  again
• not related with the actual cooperation with the
  second and the third wave
• Remark First wave respondents unwilling to be
  contacted again were excluded for the analysis of
  actual behavior

14

• Interest in the topic
• indicator1 frequency that respondents used the
  Internet to read about the news
• indicator 2 frequency that respondents
  discussed politics with friends
• respondents agreeing to be contacted in the
  future discussed politics more often and read the
  news more often
• no association with the actual behavior

15

• Utilitarian individualism
• a higher level is related with a lower intention
  to participate again
• no association with actual behavior
• joint analysis
• portion of correct answers
• item non response for opinion questions
• respondents evaluation of survey agreeability
• utilitarian individualism

16
(No Transcript)
17

• Actual co-operation in the second and third wave
• the respondents evaluation of the first
  interview has a significant effect on the
  participation in the second wave
• proportion of correct responses has a significant
  positive effect on the participation in the third
  wave (same topic)

18
4. Conclusion

• Panel data are useful to model unit non response
• Same variables have an effect on unit non
  response in face-to-face panels and web surveys
  panels
• The same basic decision mechanism in both modes
  cost benefit analysis
• a more optimal design all first wave
  participants
• Same model for unit non response in general ?