The different roles of interviewers: How does interviewer personality affect respondents

1
The different roles of interviewers How does
interviewer personality affect respondents
survey participation and response behavior?

• Michael Weinhardt (DIW Berlin)
• International Total Survey Error Workshop
• Tällberg 2009

The German Socio-Economic Panel Study is a
service unit of the
2
Multiple Error Sources

• Unit Nonresponse
• No contact, no opportunity
• Lack of time and/or motivation
• Fear or maybe even shame to participate
• Different Interviewer characteristics interact
  with different error sources differently
• One characteristic may reduce one type of error
  but increase another
• ? conflicting priorities when recruiting
  interviewers

• Measurement Error
• Interviewer Characteristics
• Adherence to int. protocol
• appropriate probing
• accurate keying
• Respondent Characteristics
• Acquiescence
• Extreme respsonse style
• Item Characteristics
• Social Desirability
• Privacy

3
Survey Error and Interviewer Personality
Conscien-tiousness

• Conscientiousness
• neat, orderly and meticulous
• Extraversion
• Sociability, enthusiasm, outgoing character
• Agreeableness
• Gentle, kind, seeking consultation, not
  confrontation
• Openness to Experience
• Open for intellectual stimulation
• new and unconventional ideas
• Neuroticism
• Emotional instability, nervousness

Ints performance
-

Extraversion
Privacy Effects
-
Satisficing
Agreeable-ness
-
Social Desira-bility Effects
-
Openness
Unit Nonresponse
Neuroticism
4
Data 1 German Socio-Economic Panel Study

• German PSID Nationally representative
  longitudinal panel study of private households
  and individuals
• Annual since 1984 12,000 households 22,000
  persons
• Two-step sampling design
• Register sampling of communities (polling
  districts)
• Random route procedure
• Mixed Modes
• Personal Interviewing whenever possible (25
  PAPI, 25 CAPI)
• 36 self-administered (Interviewer present)
• 14 postal

5
Data 2 GSOEP Interviewer Survey

• Mail survey of all interviewers of the 2006 SOEP
  wave
• Response rate 94 N552 interviewers
• Update on interviewers demographics and other
  characteristics
• Systematic feedback on interviewers work
  experience motivation, workload, opinion on
  incentives etc
• Self-rated measures of attitudes, values, beliefs
  etc.
• Same question format as in SOEP questionnaires
  regular for respondents
• Data linkage to individual level respondent data
  of the 2006 wave
• Measure of personality Shortend Big Five
  Inventory (BFI)
• Also in SOEP 2005 ? for both respondents and
  interviewers available

6
Interviewers Big Five
7
Analysis Model Specification

• Two level hierarchical regression models,
  face-to-face interview cases only
• Dependent variables Measurement Error
  indicators
• Independent variables Big Five variables
• Personality congruence between
  interviewer and respondent
• Interviewer level covariates Key demographics,
  experience as an interviewer, workload
• Respondent level covariates Key demographics
  (age, gender, region, education, whether
  self-employed), experience with SOEP

8
Looking at Measurement Error Results and
Challenges

• Findings
• Interviewer conscientiousness reduces overall
  item nonresponse
• Personality congruence influential across a range
  of measures
• Interviewers personality dimensions are
  significantly related to (some) indicators of
  measurement error
• But
• Difficult to hypothesize links between
  personality dimensions and measurement error
  lack in theory
• Single dimensions of interviewers personality
  show no consistent links to measurement error
  across a range of measures
• Unexpected directions of effects

9
Interviewer performance and overall data quality
  Logistic Regression Logistic Regression Negative Binominal Regression Negative Binominal Regression
Interviewer Level Covariates Overall Item NR (binary) Overall Item NR (binary) Overall Item NR (Count) Overall Item NR (Count)
Interviewer Level Covariates Odds Ratio Std. Err. Coefficient Std. Err.
Experience as Interviewer 1.002 0.010 0.000 0.003
Gender 0.767 0.103 -0.249 0.042
Region 0.791 0.257 -0.149 0.119
Age 1.015 0.007 0.013 0.002
Workload 0.997 0.002 -0.002 0.001
Openness 1.006 0.008 -0.001 0.001
Conscientiousness 0.984 0.008 0.010 0.002
Extraversion 1.012 0.008 -0.015 0.003
Agreeableness 0.997 0.009 0.002 0.003
Neuroticism 1.003 0.007 -0.007 0.003
Pers. Congruence 0.990 0.003 -0.001 0.002
of Items     0.000 0.000
Rho 0.238 0.021    
Number of Obs   8975   8975
Number of Int's   417   417
Significance 10 level 5 level
1 level
10
Significance 10 level 5 level
1 level
Privacy effects NR income items
  Logistic Regression Logistic Regression Logistic Regression Logistic Regression Logistic Regression Logistic Regression
Interviewer Level Covariates Gross Income NR (binary) Gross Income NR (binary) Net Income NR (Count) Net Income NR (Count) Salary NR (Count) Salary NR (Count)
Interviewer Level Covariates Odds Ratio Std. Err. Odds Ratio Std. Err. Odds Ratio Std. Err.
Experience as Interviewer 1.005 0.023 0.988 0.020 1.004 0.020
Gender 0.966 0.280 0.871 0.236 1.172 0.302
Region 2.823 1.803 0.941 0.576 1.454 0.909
Age 0.981 0.014 0.978 0.014 0.985 0.013
Workload 0.995 0.005 0.997 0.004 0.991 0.005
Openness 1.013 0.018 0.986 0.016 0.999 0.016
Conscientiousness 0.984 0.018 0.985 0.017 0.987 0.017
Extraversion 1.009 0.017 1.023 0.017 1.030 0.016
Agreeableness 1.015 0.020 1.019 0.019 1.014 0.018
Neuroticism 1.028 0.015 1.019 0.014 1.009 0.013
Pers. Congruence 0.992 0.007 0.980 0.008 0.982 0.009
Rho 0.507 0.146 0.388 0.142 0.333 0.049
Number of Obs   4540   4604   4224
Number of Int's   369   372   366
11
Significance 10 level 5 level
1 level
Social Desirability Alcohol, cigarettes and
weight
  Logistic Regression Logistic Regression OLS Regression OLS Regression Logistic Regression Logistic Regression
Interviewer Level Covariates Regular alcohol consumption (binary) Regular alcohol consumption (binary) Cigarettes smoked per day (continuous) Cigarettes smoked per day (continuous) Round values weight variable (binary) Round values weight variable (binary)
Interviewer Level Covariates Odds Ratio Std. Err. Coefficient Std. Err. Odds Ratio Std. Err.
Experience as Interviewer 1.001 0.009 0.017 0.017 1.001 0.005
Gender 0.897 0.099 -1.032 -1.032 1.066 0.064
Region 1.113 0.340 -0.711 -0.711 0.895 0.158
Age 1.012 0.006 -0.011 -0.011 0.998 0.003
Workload 1.001 0.002 -0.005 -0.005 0.999 0.001
Openness 0.987 0.006 0.047 0.047 1.011 0.004
Conscientiousness 0.989 0.007 0.028 0.028 1.006 0.004
Extraversion 0.993 0.006 0.024 0.024 1.005 0.004
Agreeableness 0.992 0.007 -0.049 -0.049 1.003 0.004
Neuroticism 1.001 0.006 0.009 0.009 0.996 0.003
Pers. Congruence 0.998 0.003 -0.030 -0.030 0.991 0.002
Rho 0.109 0.016 0.013   0.018 0.005
Number of Obs   8927   2463   8907
Number of Int's   416   342   417
12
Acquiescence and Extreme Response style
  OLS Regression OLS Regression Negative Binomial Regression Negative Binomial Regression
Interviewer Level Covariates Indicator Acquiescence Indicator Acquiescence Indicator Extreme Response Indicator Extreme Response
Interviewer Level Covariates Coefficient Std. Err. Coefficient Std. Err.
Experience as Interviewer -0.010 0.010 0.001 0.003
Gender -0.166 0.136 0.038 0.041
Region 0.437 0.351 0.026 0.097
Age 0.013 0.007 -0.004 0.002
Workload 0.003 0.002 -0.002 0.001
Openness 0.015 0.008 -0.004 0.002
Conscientiousness -0.005 0.009 0.010 0.003
Extraversion 0.003 0.008 0.000 0.002
Agreeableness 0.006 0.009 0.000 0.003
Neuroticism 0.019 0.007 -0.011 0.002
Pers. Congruence -0.032 0.003 0.004 0.001
Rho 0.107      
Number of Obs   8927   8975
Number of Int's   416   417
Significance 10 level 5 level
1 level
13
Attrition and unit Non-response

• Preliminary analysis same model set-up as before
• Linked 2006 data on interviewers and respondents
  to 2007 outcomes - binary variable attriter or
  not
• No significant results neither for big five nor
  for personality congruence
• However overall item missingness is
  significantly related to attrition
• Further thoughts on attrition analysis
• What kind of effect would we expect?
• Original distribution of Big 5 among interviewers
  not known self selection of interviewers
  possible
• Attrition due to personality effects may have
  happened before 2006 already

14

• Many thanks !
• mweinhardt_at_diw.de

15
Similarity in Personality

• Literature similarity in personality is linked
  to affection and friendship, e.g. personality
  congruence in married couples (Rammsttedt and
  Schupp 2009)
• Personality congruence might also play an
  important role in the interviewer-respondent
  relationship
• Measurement Euclidean distance between
  interviewers and respondents big five scores
• The sqareroot of the sum of squared differences
  in the big five factor scores
• Mean 50 Standard deviation 10
• Premultiplied by -1 for ease of interpretation