The Interviewer as an Informant about the Interview Process

1
The Interviewer as an Informant about the
Interview Process

• Geert Loosveldt
• Department of Sociology
• Catholic University of Leuven
• Belgium

2
I. Introduction

• Two components of the interviewer role
• persuading people to cooperate
• obtaining information
• Tendency to enlarge the task
• Expert Witnesses
• they observe the respondent reaction
• Expert Knowledge
• useful in quality assessment research
• methods rating forms, focus groups, interviewer
  debriefing questionnaires

3

• Comments
• Interviewers are a source of error
• Data about the interview are not error free
• Question
• Whats the quality of the interviewers
  information about the interview process
• General research objective
• evaluation of the quality of the data collected
  by means of interviewer reports

4
II. Research objectives

• The interviewer report
• Debriefing questionnaire
• The interviewers evaluation of
• interview situation
• respondent behaviour
• cognitive and communicative skills
• motivation to answer
• easy to collect but not often used

5

• Evaluation of the interviewer report
• First criterion association between the
  information form the report and aspects of
  response behaviour
• strong association ---gt positive evaluation
• Second criterion Interviewer variance in
  interviewers evaluation
• same response behaviour --gt same evaluation ---gt
  no interviewer variance
• absence of interviewer variance ---gt positive
  evaluation.

6

• Expectations
• information from the report is related to
  response behaviour
• interviewers do not evaluate the same response
  behaviour in the same way interviewer variance
• evaluation is written down at the end of the
  interview
• interviewer report is not a major issue of the
  training

7
III. Data

• Belgian General Election Study 1995
• Flemish population aged 18-75 years
• 2099 face-to-face interviews in respondents
  homes
• Response rate 65
• Questionnaire voting behaviour, political
  participation, political objectives and
  interviewer report

8
IV. Information from the interviewer reports

• Four general evaluation questions
• general not related to specific questions

9
Clarification question Did the respondent ask
for clarification ?

• Indicator of difficulties experienced by the
  respondent
• More than 10 asked often or a lot for
  clarification

10
Resistance question Did you feel any resistance
on the part of the respondent in answering some
questions ?
Resistance ----------------------- Never
57.3Almost never 28.3Now and then
11.4A lot 2.0Often
1.1 -----------------------
2088 11
Sensitive questions income question, questions
about political preferences. Almost never or
never in 85 of the interviews
11
Motivation questionHow do you judge the
motivation of the respondent
Motivation -------------------------------- Very
reserved 1.3Reserved
3.3Rather indifferent
15.3Moderately motivated 42.8Highly motivated
37.4 --------------------------------
2090
9
Less then 5 was considered reserved or very
reserved
12
Ability questionHow did you judge the
capability of the respondent to understand the
questions ?
Ability ---------------------------- Totally
inadequate 0.5Very poor 3.2Poor
8.5Adequate
36.4High 31.7Very high
19.6 ----------------------------
2087
12
For most respondents the evaluation is
positive 12 of the respondents had problems
13

• Observations
• Respondents were motivated and capable of
  performing their role as respondent
• All correlations between these four variables are
  significant ---gt
• Reliable scale interviewers general impression
  of the respondent
• high score positive evaluation
• low score negative evaluation
• four evaluative variables and general scale are
  used as dependent variables

14
V. Response behaviour and the interviewer
evaluation of respondents role performance

• An interviewer expects substantive and consistent
  answers---gt
• Item non-response and inconsistent answers
• can be considered as negative ---gt
• will influence interviewers evaluation
• examination of the effect of
• item non- response for difficult questions,
  attitude questions and sensitive questions
• inconsistent answers

15
Description of the difficult questions

• A rating task six political parties on 8
  different 11-point scales (6x8 questions).
• A dont know filter in the introduction but not
  on the show card.
• 20 used DK answers at least 24 times out of 48.
• Answering these questions requires a lot of
  cognitive effort and motivation.
• Expectation
• an effect on interviewers evaluation of
  respondent motivation and ability.

16
Description of attitude questions

• An attitude scale on ethnocentrism with 13 items
  (5-point scale)
• A no opinion filter in the introduction but
  not on the show card.
• Fewer than 2 gave more than 6 no opinion
  answers.
• no opinion answer is an expression of limited
  motivation.
• Expectation
• an effect on the evaluation of the respondents
  motivation

17
Description of sensitive questions

• The income questions and the political
  preference questions.
• 10 of the respondents refused to answer both the
  open and the closed income questions.
• Question about the political party the
  respondents had voted and the most important
  reason for voting for this party. For 7 no
  information.
• Expectation
• an effect on the interviewers evaluation of the
  respondent resistance to answering some
  questions.

18
Description of inconsistent answers

• Four pair of questions for which the answers can
  be interpreted as being inconsistent or not.
• 11.7 of the respondents gave an inconsistent
  answer for one or more pairs of statement. --gt
• inconsistent answers are not dominant
• Expectation
• A moderate effect on the interviewers evaluation
  of respondents motivation and ability

19
(No Transcript)
20
VI. Analysis of the effects of response behaviour
on the interviewers evaluation

• A regression analysis for each interviewers
  evaluation variable and the general scale.
• Independent variables
• five aspects of response behaviour and
• two background characteristics education and
  sex.

21
- Background characteristics have sign. effects -
All expected effects are significant. More
effects are sign. - Interviewers evaluations are
more a reflection of a general picture of the
respondent
22
VII. Analysis of interviewer variance

• Second evaluation criterion analysis of
  interviewer variance for the evaluative variables
  
• General idea
• controlling for relevant respondent
  characteristics, there should be no variation
  between interviewers with respect to the
  evaluative variables
• Multi-level models
• respondents are level-1 units
• interviewers are level-2 units

23
Model A Only the interviewer as independent
var. Differences between
interviewers are sign. High
intraclass correlations. Model B Model A and
response behaviour characteristics.
Five interviewer variances are still
significant Conclusion a negative evaluation

24
Extension of model B random intercept and random
slopes
Only a few random slopes are significant. The
impact of item non-response for the income and
political preference questions on the evaluation
of resistance is not the same
25
VII Discussion

• The interviewer is a rather reliable informant
• Question 1 How can we get better information ?
• More training to complete this part of the
  questionnaire
• differences between interviewers reflect
  differences in the way interviewers react.
• Question 2 How can we use the information ?
• Typology of respondents
• use the typology in a substantive analysis