EDUC 6033

1
EDUC 6033

• Research Design and Analysis
• Week 6

2
Internal Validity

• The changes that are measured in the dependent
  variable result from the manipulation of the
  independent variable and not something else (an
  extraneous variable).

3
Major Threats to Internal Validity

• History
• Maturation
• Testing
• Instrumentation
• Statistical Regression
• Selection
• Mortality

4
History

• Specific events that occur between the first and
  second measurements.
• Example
• A government teacher gave his students a survey
  on August, 2001 and a second survey in December,
  2001 to assess their patriotism. He concludes
  that his course caused his students to view the
  US differently.

5
Maturation

• Changes within the subjects resulting from the
  passage of time not specific to the particular
  events), including growing older, growing
  hungrier, growing more tired, and the like.
• Example
• A superintendent believes that the increase in
  the number of discipline problems between
  elementary and middle schools to be the result of
  poor principalship in the middle schools.

6
Testing

• The effects of taking a test upon the scores of a
  second testing.
• Example
• The average student who takes the SAT a second
  time increases his or her scores by approximately
  30 points however, that is the average student.
  http//www.collegeboard.com/satscores/letter.html

7
Instrumentation

• Changes in the calibration of a measuring
  instrument or changes in the observers or scorers
  used may produce changes in the obtained
  measurements.
• Example
• In a study to measure changes in aggressive
  behavior on a school yard, an initial count of
  such behaviors was determined by teacher A in
  January and Teacher B in June.

8
Statistical Regression

• Extreme scores often are anomalous. Especially
  when groups have been selected on the basis of
  their extreme scores.
• Example
• Students who take a test for the first time
  sometimes get lucky and do exceptionally well,
  while others have a bad day and do very poorly.

9
Selection

• Biases resulting in differential selection of
  respondents for the comparison groups.
• Example
• A teacher uses a morning class and an afternoon
  class as the experimental and control goups for a
  study of the effects of video instruction on
  attentiveness.

10
Mortality

• Differential loss of respondents from the
  comparison groups.
• Example
• An education program in a university touts the
  effectiveness of their program to increase
  motivated behaviors by comparing the
  motivated scores of entering students to those
  that graduate.

11
Weak Experimental Research Designs

• One-group posttest-only design.
• Which of the threats are controlled?

History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality (Red
Controlled)
12
Weak Experimental Research Designs

• One-group pretest-posttest design.
• Which of the threats are controlled?

History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality
13
Strong Experimental Designs

• Post-test only control-group design.
• Which of the threats are controlled?

History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality
14
Strong Experimental Designs

• Pretest-posttest control-group design.
• Which of the threats are controlled?

History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality
15
Strong Experimental Designs

• Solomon four-group design.
• Which of the threats are controlled?

History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality
16
Quasi-Experimental Designs
Nonequivalent Comparison Group Design
History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality

• Which of the threats are controlled?

17
Quasi-Experimental Designs
Interrupted Time-Series Design
History Maturation Testing Instrumentation Statist
ical Regression Selection Mortality
Which of the threats are controlled?