Questionnaire Design

1
Questionnaire Design
2
It is not every question that deserves an
answer.

• Publius Syrus(Roman, 1st century B.C.)

3
Ask a Sensitive Question, Get a Sensitive Answer

• Survey researchers believe that the responses
  that people give are valid.
• Care must be taken with sensitive questions.
• Researchers must take care in asking relevant
  questions in ways that produce the most truthful
  results.

4
Basic Considerations in Questionnaire Design

• Questionnaire design is one of the most critical
  stages in the survey research process.
• A questionnaire (survey) is only as good as the
  questions it asks ask a bad question, get bad
  results.
• The questions must meet the basic criteria of
  relevance and accuracy.

5
What Should Be Asked?

• Questionnaire Relevancy
• All information collected should address a
  research question that helps the decision maker
  in solving a current marketing problem.
• Questionnaire Accuracy
• The information is valid it faithfully
  represents reality.
• Questionnaires should use simple, understandable,
  unbiased, unambiguous, nonirritating words.
• Questionnaire design should facilitate recall
  motivate respondents to cooperate.
• Proper question wording sequencing to avoid
  confusion biased answers.

6
Major Decisions in Questionnaire Design

• What should be asked?
• How should each question be phrased?
• In what sequence should the questions be
  arranged?
• What questionnaire layout will best serve the
  research objectives?
• How should the questionnaire be pretested? Does
  the questionnaire need to be revised?

7
Phrasing Questions

• Open-ended questions
• Aka essay questions, short-answer questions
• Fixed-alternative questions
• Aka closed or closed-ended questions

8
Open-Ended Response Questions

• Pose some problem ask respondents to answer in
  their own words.
• Advantages
• Particularly beneficial in exploratory research,
  especially when the range of responses is not
  known.
• Identify which words phrases people
  spontaneously give.
• Valuable at the beginning of an interview.
• Disadvantages
• High cost of administering open-ended response
  questions.
• The possibility that interviewer bias will
  influence the answer.
• Bias introduced by articulate individuals longer
  answers.

9
Example of Open-Ended Response Question
10
Fixed-alternative Questions

• Questions in which respondents are given
  specific, limited-alternative responses asked
  to choose the one closest to their own viewpoint.
• Advantages
• Require less interviewer skill
• Take less time to answer
• Are easier for the respondent to answer
• Provides comparability of answers
• Disadvantages
• Researcher may be unaware of all potential
  responses
• Tendency of respondents to choose more
  prestigious or socially acceptable alternative

11
Example of a Fixed-Alternative Question
12
Types of Fixed-Alternative Questions

• Simple-dichotomy (dichotomous) Question
• Requires the respondent to choose one of two
  alternatives (e.g., yes or no).
• Example
• Did you make any calls with your home (landline)
  phone during the 7 days?
• _____ Yes _____ No
• Determinant-Choice (multiple-choice) Question
• Requires the respondent to choose one response
  from among multiple alternatives (e.g., A, B, or
  C).
• Example

13
Types of Fixed-Alternative Questions (cont.)

• Frequency-determination Question
• Asks for an answer about general frequency of
  occurrence (e.g., often, occasionally, or never).
• Checklist Question
• Allows the respondent to provide multiple answers
  to a single question by checking off items.
• Scale
• Likert, Semantic Differential, Stapel, etc.

14
Phrasing Questions for Self-Administered,Telephon
e, Personal Interview Surveys

• Influences on Question Phrasing
• Means of data collection telephone interview,
  personal interview, self-administered
  questionnaire will influence question format
  question phrasing.
• Questions for mail, Internet, telephone surveys
  must be less complex than those used in personal
  interviews.
• Questionnaires for telephone personal
  interviews should be written in a conversational
  style.

15
Best Question Formats Vary by the Interview
Medium
16
Guidelines For Avoiding Mistakes

• Simpler is better
• Avoid leading loaded questions
• Leading question directs respondents to an
  answer you want them to give
• Loaded question suggests a socially desirable
  answer or is emotionally charged.

17
Guidelines For Avoiding Mistakes (cont.)

• Avoid ambiguity Be as specific as possible
• Probably /Definitely Sometimes/Always, etc.
• Avoid double-barreled items
• Double-barreled question may induce bias
  because it covers two or more issues at once.
• Do you think the President is responsible for the
  federal government shut-down and the currently
  rising gasoline prices? Yes No
• Avoid making assumptions
• Given Macys skill-level at gift wrapping, .
• All-inclusive response alternatives
• Avoid taxing respondents memory

18
Avoid Common Wording Mistakes in Questionnaire
Design
19
Order Bias

• Question Sequence
• Order bias
• Bias caused by the influence of earlier questions
  in a questionnaire or by an answers position in
  a set of answers.
• Funnel technique
• Asking general questions before specific
  questions in order to obtain unbiased responses.
• Randomized Presentations
• Used in electronic questionnaires, but rarely
  used in printed questionnaires due to coding
  difficulties.
• Randomized Response Techniques
• Randomly assigning respondents to answer either
  the question of interest (embarrassing) or a
  mundane unembarrassing question.

20
Survey Flow

• Survey flow
• The ordering of questions through a survey.
• Breakoff
• A respondent who stops answering questions before
  reaching the end of the survey.
• Filter question
• A question that screens out respondents who are
  not qualified to answer a second (or follow-up)
  question.
• Branching
• Directing respondents to alternative portions of
  the questionnaire based on their response to a
  filter question.

21
Survey Flow for Eurocar Tour de France
Sponsorship
22
Telephone Questionnaire with Skip Questions
23
Survey Technology

• Physical Features
• Heat map question
• A graphical question that tracks the parts of an
  image or advertisement that most capture a
  respondents attention.
• Status Bar
• A visual indicator that tells the respondent what
  portion of the survey he or she has completed.
• Prompting
• Informs the respondent that he or she has skipped
  an item or provided implausible information.
• Piping Software
• Allows question answers to be inserted into later
  questions.

24
Tracking Points of Interest Using a Heat Map
25
Illustration of Status Bar Prompts
26
Pretesting Revising Questionnaires

• Pretesting Process
• Seeks to determine whether respondents have any
  difficulty understanding the questionnaire
• whether there are any ambiguous or biased
  questions.
• Preliminary Tabulation
• A tabulation of results of a pretest to help
  determine whether questionnaire will meet the
  objectives of the research.

27
I Give Up!

• Large portion of respondents give up before
  finishing abandon the survey (break-offs).
• Guidelines
• Visually appealing easy to read
• Fewer questions per page (no more than 20)
• 4 pages maximum for consumers
• 6 pages maximum for business leaders
• Question order (funnel vs. reverse-funnel)
• Sensitive questions open-ended questions
  encourage break-offs /or item non-response
• Sophisticated samples increase response rate
• Make information requests legitimate
• Pretesting is important (pretest EVERYTHING)

28
Questionnaire Reproduction(if mail survey)

• Professional appearance
• Booklet format for long questionnaires
• Place directions as close to questions as
  possible
• Expect reproduction errors