Survey Research Chapter 17: How To Design And Evaluate Research In Education

1
Survey ResearchChapter 17 How To Design And
Evaluate Research In Education

• James Blackwood
• AED 615 Fall Semester 2006

2
Survey Research Topics

1. What Is A Survey?
2. Why Are Surveys Conducted?
3. Types of Surveys
4. Correlational Research
5. Steps In Survey Research
6. Nonresponse
7. Problems With The Instrument
8. Evaluating Threats To Internal Validity
9. Data Analysis
10. Journal Article Employing Survey Research

3
What Is A Survey

1. Gathering of a sample of data or opinions
   considered to be representative of a whole group
   or population.
2. Information is collected primarily by asking
   questions.
3. Information is collected from a sample (or
   portion) of a population.

4
Why Are Surveys Conducted?

1. Surveys are primarily used to describe
   characteristics of a population.
2. Researchers use surveys to examine the
   distribution of the identified characteristics
   within the population.
3. The description of the population as a whole is
   inferred by the results obtained from the sample.

5
Useful Surveys (Arlene Fink, 1995)

1. Specific, measurable objectives
2. Sound research design
3. Sound choice of population or sample
4. Reliable and valid
5. Appropriate analysis
6. Accurate reporting of results

6
Types of Surveys

• There are two major types of surveys..
• Cross-Sectional Surveys
• Longitudinal Surveys

7
Cross-Sectional Surveys

• Information collected from a sample of a
  predetermined population
• Information is collected at approximately one
  point in time
• If an entire population is surveyed, it is
  described as being a census (Like the US Census
  that is conducted every 10 years).

8
Longitudinal Surveys

• Information is collected at different points in
  time to study changes that occur over a time
  period
• Trend Study samples a population whose members
  may change over the course of the study
• Cohort Study - samples a population whose members
  do not change over the course of the study
• Panel Study survey the same sample of a
  population at different times during the course
  of the study

9
Correlational Research

• Correlation of the results of two separate
  surveys to determine if a relationship exists
• Scores or results of surveys are examined in this
  method by calculating correlation coefficients or
  by preparing contingency tables

10
Steps In Survey Research

• Define the problem
• Identify the population
• Choose the type of instrument to collect the data
• Design, construct, pilot and refine the
  instrument
• Select a representative sample
• Administer the survey
• Analyze, interpret, and communicate your findings

11
Nonresponse

• Nonresponse refers to members of a sample who do
  not respond to the instrument
• Total Nonresponse not responding to the entire
  survey/instrument
• Item Nonresponse not responding to some of the
  questions within a survey/instrument

12
Reducing Nonresponse

• Examples of methods to reduce nonresponse
• Pretesting an instrument
• Training interviewers
• Providing incentives for participation
• Contacting non-respondents using with an
  alternate form of the instrument used for testing
  for nonresponse error

13
Problems With The Instrument

• Many issues can affect the validity of an
  instrument
• Venue/Time (location or time that the instrument
  is presented)
• Question preparation (leading questions)
• External issues
• Characteristics of the data collector

14
Evaluating Threats To Internal Validity

• Four main threats to internal validity
• Mortailty removing data from lost members of a
  study
• Location threat data collection in locations
  that would affect responses
• Instrumentation defects in the instrument
  itself
• Instrument decay interviewers get tired or are
  rushed

15
Data Analysis

1. Summarize responses to draw conclusions from the
   results
2. Size of sample percentage of returns reported
3. Percentage of total sample responding for each
   item reported
4. Percentage of respondents who chose each
   alternative for each question reported

16
Journal ArticleExample of Survey Research
Public Perception of Extension (1996) JOE
34(4) Paul D. Warner , James A. Christenson, Don
A. Dillman, Priscilla Salant This article
addressed the issues of how people's
perception/awareness and use of Extension changed
over a 13-year period. Telephone surveys of the
U.S. population in 1982 and again in 1995 were
utilized as the instrument in this study.
17
Thanks For Listening!