SURVEY RESEARCH

1
SURVEY RESEARCH

• KRISTOPHER, KIM, SHELLEY AND JILLIAN

2
Getting Started

• A telephone survey says that 51 percent of
  college students drink until they pass out at
  least once a month. The other 49 percent didn't
  answer the phone. -Craig Kilborn

3
The Following Topics will be Discussed

• Theoretical Tradition and Epistemological
  Paradigm
• Most Appropriate Types of Research Questions
• Major Features of the Method
• Major Criteria for Rigour
• Strengths and Weaknesses
• Designing a Project using the Method

4
Theoretical and Epistemological Background

• In general, a survey involves the collection of
  information from a large group of people or a
  population, commonly via
• Opinion Surveys
• Political Polls
• TV Viewing Polls
• However

5
Theoretical and Epistemological Background

• our focus is on survey research, which is
  conducted to advance scientific knowledge or
  develop theory.
• Survey research is frequently utilized within the
  Social Sciences, including disciplines like
  Psychology, Marketing, and Organizational
  Behaviour.

6
Positivism

• Positivism has become a dominant institutional
  form in social research.
• However, during the 1970's and 1980's prominent
  concerns were raised about the limits of
  quantitative data and methods often associated
  with positivism including survey research
  designs.
• Eg Concerns explained.

7
Positively More Positivism

• Assumption Objective world which science can
  mirror with privileged knowledge.
• Key Focus Search for contextual and
  organizational variables which cause
  organizational actions.
• Key Theories in Paradigm Contingency theory,
  systems theory, population ecology, transaction
  cost economics of organizing, dustbowl
  empiricism, etc.
• Goal of Paradigm Uncover truth and facts as
  quantitatively specified relations among
  variables.

8
Positively More Positivism II

• Criteria for Assessing Research
• PredictionExplanation, Rigour internal/external
  validity, reliability.
• Unit of Analysis The variable.
• Research Methods and Types of Analysis
• Experiments, questionnaires, secondary data
  analysis, quantitatively coded documents.
• Quantitative Regression, Likert scaling,
  structural equation modelling.
• Qualitative Grounded theory testing.

9
Major Features of the Method

• Variables are often operationalized when
  researchers ask people questions as a way of
  getting data for analysis and interpretation (the
  questions are either asked by an interviewer, or
  written down and given to respondents for
  completion)
• 2 Forms of surveys
• Questionnaire document containing questions and
  others types of items designed to solicit
  information appropriate for analysis
• In dept Interviewing ask people questions in
  order to gather data
• Survey research is especially appropriate for
  making descriptive studies of large populations
• Data collected may be used for explanatory
  purposes

10
The Beauty of Questionnaires

• It provides a method of collecting Data by either
  asking people questions or asking them to agree
  or disagree with statements representing
  different points of view
• Used primarily in survey research, but also in
  experiments, field research and other modes of
  observations
• Questions can be open ended (respondents supply
  their own answer-similar to a short answer
  question on an exam), answers can be in the form
  of writing on paper or verbally reporting answers
  to an interviewer
• Example what is your opinion on abortion?
• It can also be closed ended (select from a list
  of answers provided)? This type of questionnaire
  is more common than open-ended ones because they
  provide greater uniformity of responses, and are
  more easily processed
• Example From a range of 1-5, how would you rate
  this hotel? With 1 being poor, and 5 as excellent

11
A Closer Look at Closed Ended Questions

• Two structural requirements for closed ended
  questions
• 1. Response categories provided should be
  exhaustive (should include all possible responses
  that might be expected-even if answers may be
  obscure to you and me). Usually researchers try
  to ensure this by adding a category such as
  other and may follow please specify
• 2. The answer category must be mutually
  exclusive respondent should not be compelled to
  select more than one? to ensure this, we must
  carefully consider each combination of
  categories, could the respondent reasonably
  choose more than one answer?
• If our survey is constructed carefully, we
  usually do not need to add an instruction to tell
  respondents to select the best answer.

12
Things to Note for whenAsking Questions

• Make Items Clear questionnaire items should be
  precise so that the respondent knows exactly what
  the researcher is asking
• Avoid Double-Barrelled Questions Researchers
  asking respondents for a single answer to
  question that actually has multiple parts
• Respondents must be competent to answer When
  asking respondents to provide information, must
  keep in mind whether they can do so reliably
• Respondents must be willing to answer Often, we
  would like to learn things from people that they
  are unwilling to share with us

13
Continued

• Questions should be relevant Questionnaire
  should be relevant to most respondents
• Shorts Items are the best respondents are often
  unwilling to study an item in order to understand
  it, respondent should be able to read something
  quickly, understand the main point, and select an
  answer without difficulty, answers should not be
  misinterpreted
• Avoid Negative Items/Negative words appearance
  of a negation in a questionnaire items paves the
  way for easy misinterpretation
• Avoid Biased Items and Terms Meaning of
  someones response to a question depends in large
  part on its wording, some questions seem to
  encourage particular response more than do other
  questions

14
Three Main Methods for Administering Survey
Questionnaires

• Self administered questionnaires respondents are
  asked to complete the questionnaire themselves
  (example Mail survey), administer questionnaire
  to a group of respondents gathered at the same
  place at the same time.
• Surveys administered by interviewers in
  face-to-face encounters
• Surveys conducted by the telephone

15
Mail Distribution and Return

• Research worker can either hand deliver
  questionnaires, requesting that respondents mail
  the completed questionnaires to the research
  office or after questionnaires are mailed,
  researcher visits homes to pick them up and check
  for completeness
• Completion rate seems to be higher when research
  worker delivers the questionnaire, picks it up or
  both.
• Basic method of collecting data through mail is
  to send questionnaire, with a letter of
  explanation and a self addressed, stamped
  envelope for retuning (for example, like the
  surveys we receive in our mailboxes, many often
  have the postage paid indicated in the corner
  of the envelope)
• The Main reason for not returning questionnaires
  is that its TOO MUCH TROUBLE!

16
Acceptable Response Rates

• Response rates the number of people
  participating in a survey divided by the number
  selected in the sample (percentage form)
• Inferential statistics used with survey analysis
  assumes that all members of the initial sample
  complete and return their questionnaires
• Because that is very rare to happen, response
  bias is a concern?researchers often hopes for the
  possibility that the respondents look essentially
  like a random sample of the initial sample, and a
  somewhat smaller random sample of the total
  population

17
Continued

• Overall response rate is one guide to the
  representativeness of the sample reponsdents?high
  response rates means less chance of significant
  response bias, with low response rates ,
  correspondents are likely to differ from the
  respondents in ways other than their willingness
  to participate in the survey
• Rough guides (with no statistical basis) to what
  is a good response rate 50 is adequate for
  analysis and reporting, 60 is good and 70 is
  very good

18
Interview Surveys

• Interviewers ask questions orally and record
  respondents answers
• Usually done in a face-to-face encounter, but
  telephone interviewing follow similar guidelines
• Most interviews require more than one
  interviewers, but small scale interviews can be
  done by one person
• Researchers must assume that a questionnaire item
  will mean the same thing to every respondent and
  every given response must mean the same when
  given by different respondents
• The interviews presence should not affect a
  respondent's perception of a question or the
  answer given
• To save time and money, a given interviewer is
  typically assigned to complete all the interviews
  in a particular geographical area

19
General Guidelines for Survey Interviewing

• Appearance and Demeanor interviewers should
  dress in a fashion similar to that of the people
  theyll be interviewing (cleanliness and neatness
  in modest apparel). Dress and grooming are
  typically regarded as signs of a persons
  attitudes and orientations. Interviewers should
  be pleasant, must communicate interest in getting
  to know the respondent without appearing to spy
  (never too casual or clingy). Interview will be
  more successful if the interviewer can become the
  kind of person the respondent is comfortable
  with, respondents deserve the most enjoyable
  experience the researcher can provide

20
Continued

• Familiarity with Questionnaire must be able to
  read questionnaire items to respondents without
  error and stumbling over words, lines must be
  read as though they are part of a natural
  conversation
• Interviewer must be familiar with the
  specifications prepared in conjunction with the
  questionnaire
• Some questions will not fit a given respondent's
  situation, interviewer must determine how the
  question should be interpreted in that situation
• It would be better for the interviewer to leave a
  question unanswered than to spend a period of
  time searching though the specifications for
  clarifications or trying to interpret the
  relevant instructions
• Following Question Wording Exactly
• Recording Responses Exactly
• Probing for Responses Sometimes respondents
  will give an inappropriate/incomplete answer,
  request for an elaboration can be useful (Probe).
  Probes must be completely neutral, must not
  affect the nature of subsequent response

21
Advantages of Doing an Interview Survey

• They typically attain high response rates (a
  properly designed and executed interview survey
  ought to achieve a completion rate of at least
  80-85)
• Presence of and interviewer decreases the number
  of dont knows and no answers, since
  clarification can easily be done on the spot
• Interviewers can clarify matters if respondent
  clearly misunderstands the intent of the question
• Can observe respondents as well as ask questions

22
Telephone SurveysPositive Factors ()

• Saves both money and time
• May dress anyway you please without affecting the
  answers respondents give
• Respondents may be more honest in giving socially
  disapproved answers (no eye to eye contact)
• Interviewers can communicate a lot about
  themselves over the phone, even though they cant
  be seen
• Allow greater control over data collection if
  several interviewers are engaged in the project

23
Telephone SurveysNegative Factors (-)

• Some phone numbers are unlisted (but this has
  been erased through a technique called random
  digit dialling)
• Bogus surveys-ones that are actually sales
  campaigns disguised as research
• The ease in which people can hang up
• Answering machines (using machines to screen
  calls)?however research has showed that this had
  not yet had a significant effect on the ability
  of telephone researchers to contact prospective
  respondents

24
Comparing theThree Different Methods
25
Computer Assisted Telephone Interviewing (CATI)

• Increasingly used by academic, government and
  commercial survey researchers
• Central computer is programmed to select a
  telephone number at random and dials it
• Interviewer introduces study and asks the
  question displayed on the screen and then types
  that answer into the computer terminal (depending
  on the question-open/close ended)
• Computer automatically prepares the data for
  analysis, or researcher can begin analysing data
  before the interview is complete (gaining and
  advanced view of how the analysis will turn out)

26
New Technologies andSurvey Research

• CAPI (Computer assisted personal interviewing)
  face to face interviews rather than over the
  phone
• CASI (Computer assisted self interviewing)
  respondent reads questions on the computer screen
  and enters his/her own answers
• CSAQ (Computerized self administered
  questionnaire) respondent receives the
  questionnaire on a floppy disk, bulletin board,
  or other means and completes the questions,
  software then accepts the answers. Respondent
  then returns the data file
• TDE (Touchtone data entry) initiates process by
  calling a number at the research organization,
  prompts a series of computerized questions,
  respondent answers by pressing the keys on the
  telephone pad
• VR (Voice recognition) same as TDE, but the
  system accepts spoken responses
• Internet and world wide web

27
Dos and Donts for Conducting Online Surveys

• DO
• Consistent wording between the invitation and the
  survey
• Use plain simple language
• Offer to share selected results from others who
  also have completed the survey
• Plan the time of day and day of week to mail
  (when will respondents most likely be reading
  mail at home?)
• Be aware of technical limitations (Will
  respondents have programs needed to access)
• Test incentives, rewards and prize drawings to
  determine the optimal response
• Limit studies to gt15 minutes

• DO NOT
• Use terms such as unique ID number in the
  invitation, then ask respondents to type
  password when they get to the survey
• Force the respondent to scroll down the screen
  for the URL for the study location

28
Secondary Analysis

• Survey research involves 3 steps questionnaire
  construction, sample selection, and data
  collection (interviewing/self administered
  questionnaires)
• This is a form of research in which the data is
  collected and processed by one researcher are
  reanalyzed- often for a different purpose, by
  another
• Example General Social Survey (GSS)
• Advantage cheaper and faster than doing original
  surveys, benefit from the work of top flight
  professionals, enhance possibility of meta
  analysis (researcher brings together a body of
  past research on a topic)
• Disadvantage question of validity (have no
  assurance that data collected will be appropriate
  for you research interests)

29
Rigour in Survey Research

• Internal Validity
• To be internally valid, the conclusions of the
  research must be supported by the data.
• Internal validity is judged according to the
  accuracy with which a description of particular
  events represents the data.
• The essence of internal validity for survey
  research is complete confidence that your
  conclusions come from the data.

30
Reliability

• Reliability
• Reliability is generally concerned with
  replication an account is considered to be
  reliable if the data are reproducible.
• If the analytic strategy were repeated by the
  same or different investigator, then the results
  should be the same.

31
Another Type of Validity

• External Validity
• External validity or generalizability for survey
  research is captured by the question, How can
  one determine the extent to which the findings of
  a particular inquiry have applicability in other
  contexts or with other subjects?
• Essentially, the extent to which the effect can
  be generalized to populations, settings,
  treatment variables, and measurement variables.

32
External Validity Continued

• This can also refer to fit or the degree to
  which the audience or reader of the report is
  able to transfer the research findings to
  contexts outside of the study situation to other
  settings.
• The researcher must supply a substantial amount
  of clear and detailed information or thick
  description about the issue/phenomenon studied
  and the setting in which that issue/phenomenon
  was found.
• The degree of transferability is a direct
  function of the similarity or fittingness
  between the two contexts.

33
Strategies for Ensuring Rigour

• Verification Strategies
• The most important way to ensure that research is
  rigourous is to focus on verification during the
  study.
• Verification is the process of checking,
  confirming, making sure, being certain.
• Doing this as the study is conducted is key as
  the researcher then can identify and correct
  threats to reliability and validity as they
  surface.

34
Initial Strategies Address

• Investigator Responsiveness
• Methodological Coherence
• Sampling
• Data Analysis
• Thinking Theoretically

35
Additional Strategies

• There are other strategies that can be used
  during the research to contribute to rigour,
  namely
• Prolonged Engagement
• Participant Checks
• Journal Writing
• Peer Review
• Audit Trail.

36
Weaknesses for Survey Research

• Lacks the context of social life
• Appear superficial on coverage of complex topics
• Subject to artificiality
• Surveys can also be inflexible because the
  initial study design must remain the same
• Therefore, surveys are weak on validity

37
Strengths for Survey Research

• Describes the characteristics of a large
  population
• Allows a large sample of respondents
• Surveys are flexible, allows flexibility in your
  analyses
• Surveys have a strength in measurement
• Helps determine unemployment rates, and voting
  intensions
• Helps examine official documents such as
  marriage, birth, or death records
• Therefore, surveys are strong on reliability

38
Survey Design

• 1. Purpose
• 2. Respondent Group
• 3. Questionnaire Construction
• 4. Administration of Questionnaire
• 5. Analysis
• Example Sample Questionnaire (Handout)
• Student Opinion Survey on Group Assignments in
  University Courses

39
Format Guidelines

• Introduction
• Spacing
• Explanations
• Order of items
• Pre-testing

40
Additional Resources

• Presentation Website
• http//members.shaw.ca/kristopher.skinner/soc315
• Group Project E-mail
• soc315project_at_hotmail.com