Using eInterviewers Improves Web Survey Data Quality - PowerPoint PPT Presentation

Loading...

PPT – Using eInterviewers Improves Web Survey Data Quality PowerPoint presentation | free to view - id: 7efaa-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Using eInterviewers Improves Web Survey Data Quality

Description:

Karl G. Feld humanvoice inc. Steven A.Wygant Brigham Young University. www.humanvoice.com ... Designating or pertaining to a relationship between two entities, ... – PowerPoint PPT presentation

Number of Views:17
Avg rating:3.0/5.0
Slides: 31
Provided by: Asatisfied187
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using eInterviewers Improves Web Survey Data Quality


1
Using e-Interviewers Improves Web Survey Data
Quality
  • by
  • Karl G. Feld humanvoice inc.
  • Steven A.Wygant Brigham Young University

2
  • dyadic from the Greek duadikos. Designating or
    pertaining to a relationship between two
    entities, or a predicate expressing such a
    relationship. Meaning originating in the late
    nineteenth century.
  • The Oxford English Dictionary, 1993

3
The Dyadic Relationship Improves Data Quality
  • Bourque, L.B. and E. P. Fielder. 1995. How To
    Conduct Self-Administered and Mail Surveys.
    Thousand Oaks, CA Sage Publications, Inc
  • Conrad, F.G. and M. F. Schober. 2000. Clarifying
    Question Meaning In A Household Telephone Survey.
    Public Opinion Quarterly. 641-28.
  • Krosnick, J. 2000. Peering Into The Future of
    Thinking and Answering A Psychological
    Perspective on Internet Survey Respondents.
    Presentation at the 2000 Gallup Nebraska
    Symposium on Survey Research. 13-15 April.
    Lincoln, Nebraska.
  • de Leeuw, E. and M. Collins. 1997. Data
    Collection Methods and Survey Quality An
    Overview. Survey Measurement and Process Quality.
    New York John Wiley Sons, Inc.
  • Saris, W. E. 1991. Computer-Assisted
    Interviewing. Newbury Park, CA Sage
    Publications.

4
  • computer assisted methods do not provide a
    panacea (or even a general palliative) for survey
    non-coverage, non-response, and measurement
    error. Nor do they free the survey manager and
    survey designer from a need to understand the
    broader survey literature on questionnaire
    design, field work procedures, and process
    quality.
  • Nicholls, Baker and Martin, 1997

5
Why Use e-Interviewers?
  • 1. Reduce mid-termination or dropout rates
    significantly.
  • 2. Increase open-end response length.
  • 3. Screen out ineligible and professional
    respondents.
  • 4. It is inexpensive.

6
University Alumni Publications Study
  • Survey measured attitudes of alumni about
    university publications they receive.
  • Split mode methodology was used to collect
    comparative data.
  • Self administered online survey
  • e-Interviewer administered online survey
  • Interviewer administered CATI

7
Technologies Control Data Quality Across Modes
  • Integrated CATI and web survey software to
    allowed uniform, simultaneous programming.
  • List Rotations
  • Complex Skip Patterns
  • Forced Responses
  • Prevent Order Effects
  • Respondent Privacy

8
Technologies Control Data Quality Across Modes
  • Internet versions hosted in-house to guarantee
    respondents server access 24/7.
  • Servers were tied directly to the Internet
    through a Tier-1 provider.
  • Servers were redundant and load-balanced to
    prevent overload during peak periods.

9
(No Transcript)
10
Quotas Based On Winter 2000 U.S. Internet
Population
  • Age
  • Census Region
  • Gender
  • Income
  • Occupation

11
Sampling From Known Universe
  • CATI VERSION
  • Entire known universe (141,337 names) used.
  • Selected every nth point based on geographic,
    gender and age quotas.
  • INTERNET VERSIONS
  • Those with e-mail addresses not contacted by
    telephone (11,892 names) used.
  • Separated into demographically equivalent
    files, one for each version.
  • Every nth name selected based on geographic,
    gender and age quotas.

12
Data Collection Procedures
  • CATI
  • Interviewer hours evenly divided by gender.
  • No interviewer worked two days sequentially.
  • Daily quotas by quota area using sample
    proportional to final quotas.
  • Dialed sample from 9AM to 8PM after initial
    evening attempt.
  • Callbacks were scheduled upon request.
  • e-Interviewer Web
  • Interviewer hours evenly divided by gender.
  • No interviewer worked two days sequentially.
  • E-mail invitations with survey URL (and
    reminders) sent out daily in equally-sized
    batches proportional to final quotas.
  • Interviewers available 24/7.
  • Respondents could exit and re-enter surveys.

13
Data Collection Procedures
  • Self-Administered Web
  • E-mail invitations with survey URL (and
    reminders) sent out daily in equally-sized
    batches proportional to final quotas.
  • Survey available 24/7.
  • Pop-Up Exit Survey on Terminations
  • Respondents could exit and re-enter surveys.
  • e-Interviewer Web
  • E-mail invitations with survey URL (and
    reminders) sent out daily in equally-sized
    batches proportional to final quotas.
  • Survey available 24/7
  • Pop-Up Exit Survey On Terminations
  • Respondents could exit and re-enter surveys.
  • Interviewer hours evenly divided by gender.
  • No interviewer worked two days sequentially.

14
Select Final DispositionsCalculated Using
Council of American Survey Research Organization
(CASRO) and American Association for Public
Opinion (AAPOR) Standards
15
1. Significantly reduce mid-termination or
dropout rates.
16
Mid-Termination Rates Across Modes
17
Comparative Mid-Term Rates
18
Established Causes of Mid-Termination Solved By
Using e-Interviewers
  • No rapport with interviewer
  • Presence of open-end questions
  • Respondent technical difficulties
  • No opportunity to request question clarification
    or verification of sponsor
  • No option for respondent to change interview mode
    at will

19
Solving Some Knotty Internet Challenges
  • Reducing Mid-Terms Decreases Sample Panel Fatigue
    and Self-Selection Bias
  • Click to Callback and Instant Messaging Allow
    Resolution of Technical Problems Or Interview
    Mode Change
  • Increase by 30 the number of respondents 55
    years who complete surveys

20
2. Increase open-end response length.
21
Findings Consistent With Previous Research
  • Fewer terminations on open-end questions when
    interviewer present
  • Average word length of open-end responses and
    follow-up probes is longer

22
Average Open-End Word Length
23
3. Screen out ineligible and professional
respondents.
24
The Harsh, Researched Truth
  • Frequency of professional respondents in the
    U.S. is on the rise
  • Self-administered surveys have always suffered
    from ineligible respondents
  • Self-selection rates are high in sample panels
    with high mid-termination rates

25
www.money4surveys.com
26
Ineligible Respondents Caught By Version
  • CATI n300 5 ineligibles
  • E-Interviewer n117 2 ineligibles
  • Self-Administered n122 ?????????
  • Security required unique PIN and year of birth.

27
4. It is inexpensive!
28
So What Does It Cost?
  • n600
  • 50 questions
  • Listed, 80 incidence sample
  • Re-invite after initial sample send
  • Using SurveyGuardian 24/7

CATI 11356 USD
E-Interviewer 7200 USD
Self-Administered 4550 USD
29
Why Use e-Interviewers?
  • 1. Reduce mid-termination or dropout rates
    significantly.
  • 2. Increase open-end response length.
  • 3. Screen out ineligible and professional
    respondents.
  • 4. It is inexpensive.

30
Using e-Interviewers Improves Web Survey Data
Quality
  • by
  • Karl G. Feld humanvoice inc.
  • Steven A.Wygant Brigham Young University
About PowerShow.com