Learning on Campus and Learning at a Distance: A Randomized Instructional Experiment

1
Learning on Campus and Learning at a Distance A
Randomized Instructional Experiment

• A Summary by
• Ryan Nivens for
• Dr. Summers ESC PS 8020

2
Background

• NCES (1997) 60 of 2- and 4-year colleges offer
  distance learning courses
• Authors cited 12 studies suggesting benefits of
  DL
• DL and student learning studies have been flawed
  by internal validity interaction effects between
  self-selection and course achievement

3
Background

• Machtmes Asher (2000) meta-analysis of 700
  studies
• Only 2 were randomized, experimental
• Carpenter Greenhill (1963)
• Face-to-face more effective
• Ritchie Newby (1989)
• DL more effective
• Neither study reported details of instruments
  used, both were before the telecommunication
  EXPLOSION!

4
Purpose

• Address internal validity problems of past
  studies by randomly assigning students into
  groups (i.e. face-to-face or telecourse)
• Determine the effects of self-selection using a
  quasi-experimental study utilizing a third group
  (self-selected telecourse)

5
Method

• Subjects 37 students (experimental study)
• N19 students assigned to face-to-face
• N18 assigned to telecourse session
• Both met at the university
• Subjects 9 students
• Self-selected telecourse at various sites in Iowa

6
Procedure

• Variables for control (covariates) pretest
  post-secondary credit hours previous fire
  science credits licensed EMT certified fire
  fighter
• Independent variable instructional format
• Dependent variable posttest score

7
Procedure

• ANCOVA
• Student posttest score regressed on the five
  variables above, as well as the dummy coded
  instructional format
• a.10
• Preliminary test of interaction between
  covariates and instructional format
• Not significant

8
Analysis (Experimental)

• ANCOVA
• The five covariates explain 58.2 of the variance
  in posttest score (plt.01)
• Instructional format explains an additional 8.8
  (plt.05)
• Of the two randomly assigned groups, there was no
  statistically significant difference (in fact,
  almost no difference at all)

9
Analysis (Quasi-experimental)

• The self-selected telecourse group had major
  advantages over
• The face-to-face group
• 6 question advantage on posttest
• 1.32 standard deviations
• The randomly assigned telecourse group
• About a 5 question advantage
• .93 standard deviations

10
Conclusions

• These findings support the argument that
  telecourse can allow students to learn as much as
  face-to-face courses
• The self-selected telecourse group findings hint
  at interactions not controlled for in this study,
  such as self-selection x change
• This further threatens existing
  quasi-experimental studies where self-selected
  distance learners are compared with self-selected
  face-to-face groups

11
Strenghts

• Random assignment
• Jodies ethics question
• Covariates accounted for 58.2 of the variance!
• Angelas question answered
• Pretest effects were controlled
• Andrews question
• Reliability measures of pre/post test
• Kuder-Richardson 21
• .75 for pretest
• .78 for posttest

12
Weaknesses

• Erikas question
• Basic fact test (2/3 factual, 1/3 conceptual)
• Required course vs. elective?
• Only covers a Fire Science course
• Small N
• 38 for final analysis, 29 for random experiment
• Ethnicities and gender
• 94 male, 94 white
• Age range
• 19 to 40