Survey Research - PowerPoint PPT Presentation

Loading...

PPT – Survey Research PowerPoint presentation | free to view - id: 1b7f9-NzMwM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Survey Research

Description:

Survey Research. Chapter 10 (JRM) Article Critique Due April 3rd. Ch. 10: Elite ... 4. SPEND LESS. 5. SPEND MUCH LESS. 6. CANT CHOOSE. Question Order ... – PowerPoint PPT presentation

Number of Views:49
Avg rating:3.0/5.0
Slides: 84
Provided by: uky
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Survey Research


1
Survey Research
  • Chapter 10 (JRM)
  • Article Critique Due April 3rd

2
Ch. 10 Elite Interviewing and Survey Research
  • Elite Interviewing the process of interviewing
    respondents in a nonstandardized, individualized
    manner.

3
Why do an elite interview vs. sending elites a
survey?
  • 1.  Researcher may not know enough about the
    issue to construct a good survey device.
  • 2.  Researcher does not want to constrain the
    responses of the elite.
  • 3.  Elite respondents may feel a standardized
    survey is beneath them.

4
Survey Research
  • Popular form of political science research (also
    called opinion polling).
  • Goal is to measure accurately peoples
    attitudes, beliefs, and behaviors by asking them
    questions

5
Survey Research
  • Two types of error exist in survey research
  • Sampling error error due to the fact that a
    sample of people are used to estimate population
    characteristics (parameters)
  • Response error error due to features of the
    survey instrument

6
Sampling Error
  • Sampling distribution collection of sample
    statistics calculated from all possible random
    samples of a given size, taken from the same
    population
  • Statistical theory tells us what that
    distribution of sample statistics looks like

7
Sampling Error (contd)
  • Margin of Error the amount of possible error
    that exists for a sample statistic (with high
    degree of certainty)
  • Example An estimate of presidential approval for
    Obama is 60 (4), with 95 certainty
  • Margin of error allows you to calculate a
    confidence interval 604(64,56)

8
Sampling Error (contd)
9
Interpreting Margin of Error
10
Response Error
  • Survey experts agree BY FAR the largest amount
    of error in survey research comes from response
    error

11
Sources of Response Error
  • Survey Administration
  • Question Order
  • Question Type
  • Question Wording

12
Survey Administration
  • There are three typical methods used to
    administer surveys
  • Interview
  • Telephone
  • Mail

13
Personal/Telephone Interviewing
  • Advantages
  • High response rate
  • Flexibility/control in questioning
  • Observation (personal interviewing)
  • Supervision of interviewers (telephone)

14
Personal/Telephone Interviewing
  • Disadvantages
  • Expensive
  • Lack of anonymity
  • Characteristics of interviewer may introduce bias

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Mail Surveys
  • Advantages
  • Cheap!!!
  • Often avoid bias due to interviewer
    characteristics / social desirability

19
Mail Surveys
  • Disadvantages
  • Low response rates
  • Cannot probe
  • Cannot clarify
  • Cannot be sure who completes questionnaire

20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Dillmans Total Design Method for Mail Surveys
  • First Mailing 24 response rate.
  • Typically, when you send a survey out by mail,
    about 24 of those you sent it to will respond.
  • Why do we care about the response rate so much???

25
Dillman Method
  • After you send the first survey, wait about two
    weeks and then send a follow-up postcard
  • The postcard should be sent only to
    non-respondents to save money
  • Disadvantage Loss of anonymity

26
Dillman Method
  • The postcard should say something like, In the
    past week or so you received a survey from me
    about XYZ. I know you are a very busy person,
    but it is important for me to include your
    opinions in my research. If you still have the
    survey, could you please complete it and send it
    to me in the stamped addressed envelope I
    included? If it got misplaced, I am happy to
    send you a replacement survey. Please call or
    email me at ABC and I will send a replacement to
    you right away. Thank you so much for helping me
    with this important project. Sincerely, Jane
    Doe.
  • The postcard follow-up typically increases
    response rate to about 42.

27
Dillman Method
  • About one to two weeks after you send the
    postcard, send the first replacement
    questionnaire
  • Attribute the lack of response to the first
    survey getting lost in the mail, accidentally
    misplaced or discarded, or lack of time, etc.
    Stress the importance of that persons response
    to your project.
  • The first replacement survey will usually get
    your response rate up to about 59.

28
Dillman Method
  • Send a second replacement survey by certified
    mail about two weeks after the first replacement
    survey was sent. This will increase the response
    rate to about 72

29
Dillman Method
  • Other pieces of advice to maximize response rate
  • Keep it simple
  • Make it look like personal mail
  • Get a sponsor
  • Offer inducements

30
The New Frontier in Survey Administration The
Internet
  • Email surveys
  • Web surveys

31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
Question Order
  • The order in which you place your questions can
    cause the error of a response set.
  • A response set may occur when a series of
    questions have the same answer choices. This is
    most likely to happen if questions are on related
    subjects.

35
Question OrderExample Response Set (from the
GSS)
  • Listed below are various areas of government
    spending. Please indicate whether you would like
    to see more or less government spending in each
    area. Remember that if you say much more, it
    might require a tax increase to pay for it.
  • A. The environment.
  • 1. SPEND MUCH MORE
  • 2. SPEND MORE
  • 3. SPEND SAME
  • 4. SPEND LESS
  • 5. SPEND MUCH LESS
  • 6. CANT CHOOSE

36
Question OrderExample Response Set (from the
GSS)
  • B. Health Care.
  • 1. SPEND MUCH MORE
  • 2. SPEND MORE
  • 3. SPEND SAME
  • 4. SPEND LESS
  • 5. SPEND MUCH LESS
  • 6. CANT CHOOSE

37
Question OrderExample Response Set (from the
GSS)
  • C. Education.
  • 1. SPEND MUCH MORE
  • 2. SPEND MORE
  • 3. SPEND SAME
  • 4. SPEND LESS
  • 5. SPEND MUCH LESS
  • 6. CANT CHOOSE

38
Question OrderExample Response Set (from the
GSS)
  • D. Unemployment Benefits.
  • 1. SPEND MUCH MORE
  • 2. SPEND MORE
  • 3. SPEND SAME
  • 4. SPEND LESS
  • 5. SPEND MUCH LESS
  • 6. CANT CHOOSE

39
Question Order
  • Etc.
  • What is the danger in a response set and what can
    we do about it???

40
Question Order
  • The order in which you place your survey
    questions can cause the error of a saliency
    effect
  • Specific mention of an issue in a survey may
    cause a respondent to associate the issue in
    connection with a later question.

41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
Question Order
  • Consistency is another problem we might encounter
    due to question order.
  • Respondents may strive to appear consistent in
    their answers an answer to a question may be
    constrained by an answer given earlier.

45
Question OrderExample Consistency
  • And what about the federal government, does it
    have too much power or too little power?
  • FAR TOO MUCH
  • TOO MUCH
  • RIGHT AMNT OF POWER
  • TOO LITTLE POWER
  • FAR TOO LITTLE POWER
  • CANT CHOOSE
  • What if this question preceded the government
    spending questions?

46
Question Order
  • A final concern about question order is fatigue
  • Respondents may give perfunctory answers to
    questions late in the survey especially if it is
    long
  • In addition, response sets are often a problem in
    lengthy surveys toward the end

47
Question Order
  • Some final thoughts on question order
  • The first few questions should break the ice
    dont jump in with a big question.
  • Move from general questions to more specific
    questions.
  • Questions on personal or sensitive topics (such
    as income, race, age) should be asked at the end
    with a big disclaimer about why you are asking
    those questions (also helps with fatigue because
    respondents dont have to think about the answers)

48
Question Type
  • Closed-ended questions are questions that have
    response options provided by the researcher.
  • What are some of the advantages and disadvantages
    to closed-ended questions?

49
Question Type
  • Advantages of Closed-Ended
  • Easy to answer
  • Answers are easy to compare
  • Respondents willing to provide more personal
    information
  • May help clarify the question

50
Question Type
  • Disadvantages of Closed-Ended
  • Forces people to accept the categories or puts
    too many responses into the other category.
  • Choices listed may communicate the kind of
    response wanted.

51
Question Type
  • Open-ended questions allow the respondent to
    write in an answer.
  • What are some of the advantages and disadvantages
    to open-ended questions?

52
Question Type
  • Advantages of Open-ended
  • May better indicate respondents
    thoughts/beliefs/opinions.
  • Better when the list of responses might be
    excessive.
  • Might get unanticipated answers.

53
Question Type
  • Disadvantages of Open-ended
  • Respondent may say too much or too little.
  • Difficult to code all the various answers and
    analyze them.
  • Can be expensive and time-consuming.

54
Question Type
  • Single-sided vs. Two-sided questions.
  • Single-sided a question that asks respondents to
    agree or disagree with a single substantive
    statement.
  • Two-sided a question that offers the respondent
    two substantive choices.

55
Question Type
  • Single-sided example
  • Do you agree or disagree with the idea that the
    government should see to it that every person has
    a job with a good standard of living?

56
Question Type
  • Two-sided example
  • Do you think that the government should see to it
    that every person has a job with a good standard
    of living, or should it let each person get ahead
    on his or her own?

57
Question Type
  • With a single-sided question a larger percentage
    of respondents tend to agree with the statement
    given.

58
Question Type
  • 44 of respondents agreed with the single-sided
    statement.
  • Only 30.3 of respondents agreed government
    should guarantee employment and standard of
    living in the two-sided question.
  • Two-sided questions reduce the proportion of
    respondents who give no opinion.

59
Question Type
  • Social Desirability a form of response error in
    which respondents are not truthful due to the
    fact that they perceive their answer to be out of
    step with the mainstream of society

60
Race Relations and the New South
61
Race Relations and the New South
  • Experimental Design
  • Random assignment of individuals into two groups
    (national random sample of white Americans)

62
Race Relations and the New South
63
Race Relations and the New South
64
Race Relations and the New South
  • Result
  • South 42 angry
  • Non-South 11 angry

65
Question Wording
  • Double-Barreled Questions are two questions in
    one. Example
  • Do you agree with the statement that the
    situation in Iraq is deteriorating and that the
    United States should increase the number of
    troops in Iraq?
  • What is wrong with how this question is worded?

66
Question Wording
  • Ambiguous Question one that contains a concept
    that is not clearly defined.
  • What is your income?
  • Do you prefer Brand A or Brand B?
  • Rate the performance of the two debaters as good,
    bad, or indifferent.
  • What is the problem with these three survey
    questions???

67
Question Wording
  • The choices you provide on a frequency scale can
    influence how respondents answer. Consider this
    question
  • How much tv do you watch per day?

68
Question Wording
  • When the response options were
  • Up to .5 hours
  • .5 to 1 hour
  • 1-2 hours
  • 2-2.5 hours
  • 2.5 hours
  • 16 responded they watched 2.5 hours.

69
Question Wording
  • When the response options were
  • Up to 2.5 hours
  • 2.5 to 3 hours
  • 3-4 hours
  • 4-4.5 hours
  • 4.5 hours
  • 38 responded they watched 2.5 hours.
  • (From Schwarz et al., 1985)

70
Question Wording
  • Leading Questions Encourages respondents to
    choose a particular response because the question
    indicates the researcher expects it.
  • Dont you think that global warming is a serious
    environmental problem?

71
Question Wording
  • Polls conducted by political organizations and
    politicians often include leading questions.
  • 1980 poll for the Republican National Committee
    read as follows on the next slide (a dated, but
    classic example that is widely used).

72
Question Wording
  • Recently the Soviet armed forces openly invaded
    the independent country of Afghanistan. Do you
    think the U.S. should supply military equipment
    to the rebel freedom fighters?

73
Question Wording
  • Word Choice may lead respondents to an answer as
    well. You have to really think about what words
    you use to describe people, things, and events.

74
Question Wording
  • Helping the needy compared to those on
    welfare.
  • Socialized Medicine vs. National Health
    Insurance
  • Fight Against Terrorism vs. Going to War

75
Question Wording
  • Linking personalities or institutions to issues
    can also affect responses.
  • Would you say that Governor Beshears program
    for promoting economic development has been very
    effective, fairly effective . . .
  • A respondents feelings about Governor Beshear
    might affect his or her response to the question
    about economic development.

76
Question Wording
  • Push Polls These are relatively new and
    conducted by campaigns, parties, and political
    organizations.
  • Interviewers feed respondents false and damaging
    information about a candidate or cause under the
    guise of asking a question.

77
Question Wording
  • Do you agree or disagree with Candidate Xs
    willingness to tolerate terrorism in our
    country?
  • The goal is NOT to conduct research but to use
    innuendo to spread rumors and lies.
  • This is a politically risky strategy and if its
    too blatant, it can back-fire.

78
Question Wording
  • Confusing Questions should be avoided. Yes, this
    seems so obvious, but even the experts make
    mistakes sometimes. Double-negatives are
    especially confusing.
  • The following question was asked of a national
    random sample by the Roper Center, a respected
    polling organization.

79
Question Wording
  • Does it seem possible or does it seem impossible
    to you that the Nazi extermination of the Jews
    never happened?
  • A) Impossible it never happened
  • B) Don't know
  • C) Possible it never happened

80
Question Wording
  • The responses were as follows
  • Impossible it never happened 65
  • Don't know 12
  • Possible it never happened 22

81
Question Wording
  • Does it seem possible to you that the Nazi
    extermination of the Jews never happened or do
    you feel certain that it happened?
  • A) Certain it happened
  • B) Don't know
  • C) Possible it never happened

82
Question Wording
  • The responses were as follows
  • Certain it happened 91
  • Don't know 8
  • Possible it never happened 1

83
The Polls and the 2008 Presidential Election
  • http//www.pollster.com/polls/us/08-us-pres-ge-mvo
    .php
  • http//www.pollster.com/blogs/the_art_and_science_
    of_choosin.php
  • http//pewresearch.org/pubs/964/cell-phones-and-th
    e-2008-vote-an-update
About PowerShow.com