EDC

1
EDCI 510History of Educational Technology

• July 5, 2005
• The Foundations of Educational Research

2
Topics for Today

• Your Questions
• The Computer Model of Cognition
• Courtneys presentation on Richard Clark
• Bobbis presentation on Bob Kozma
• Problems with Educational Research
• No Significant Difference
• Other Issues from Chapters 8 15

3
The Computer Model of Cognition
4
The Great Delivery Truck Debate
5
Reconsidering Research on Learning from
Media--Richard Clark

• This debate has been contested ever since Clark
  published this article.
• Why?

6
Reconsidering Research on Learning from Media

• Articles primary focus is on media selection
• Media do not influence learning under any
  conditions
• What we have is research with media rather than
  research on media
• Confounding is often found in the studies
• Comes from method or content differences or
  novelty effect

7
Reconsidering Research on Learning from Media

• Should not confuse teacher with teaching
• There is also a problem in media attributes
• Attributes are an integral part of media
• Attributes will cultivate cognitive skills

8
Other Clark ArticlesConfounding in Educational
Computing ResearchJ. Educational Computer
Research, Vol. 1 (2), 1985

• Most media research is confounded
• Any effect is attributed to
• Different instructional methods
• Content
• Novelty
• This diminishes the role of computing research in
  developing an instructional theory but not in ID
  or delivery

9
Other Clark ArticlesDangers in the Evaluation
of Instructional MediaAcademic Medicine Vol 67,
No. 12, Dec. 1992

• I have argued that achievement gains result from
  the use of instructional methods that are
  appropriate for learning objectives and students
  aptitudes. Methods such as examples, analogies,
  models, interactive simulations, practice, and
  feedback, when applied appropriately, can
  significantly enhance learning.
• Introduction to PlanAlyzer, an Interactive
  Computer-assisted Program, Lyon, et. al. in
  which Lyon used a Cross-Over Design with medical
  students studying Hematology and Cardiology.

10
Learning With MediaGreg Kozma

• Media do influence learning
• Capabilities of media and effective methods,
  working together, are the answer
• Media is defined by
• Its technology
• Its symbol system
• The cognitive processes that are elicited by it.

11
Learning With Media

• Learning is viewed as an active, constructive
  process where the learner strategically manages
  the available cognitive resources to create new
  knowledge by extracting information from the
  environment and integrating it with information
  already stored in memory.
• Thus symbol systems are not sufficient to
  describe a medium and its cognitive effects. You
  must also include information that is being
  processed in memory.

12
Learning With Media

• Amount of invested mental effort is important. It
  is affected by
• Attitudes people have about the amount of effort
  required to process the message in the media
• The cognitive loadhow hard the learning tasks
  are to the learner
• Viewer control is an important instructional
  design consideration

13
Learning With Media

• The computer medium
• Offers self-pace
• Learner can manipulate entities within the
  computer program
• Hypertext facilitates the application and
  transfer of complex knowledge to new situations.
• This flexibility requires the presentation of
  knowledge along multiple rather than single
  conceptual dimensions.

14
For More Information

• Learning from Media Arguments Analysis, and
  EvidenceEdited by Richard E. Clark

15
Effectiveness of Computer-Based Instruction An
Updated AnalysisKulik Kulik

• Meta-analysis involves 5 steps
• Defining Inclusion Criteria
• Locating Studies
• Coding Study Features
• Quantifying individual study outcomes
• Analyzing Data

16
Meta-Analysis Method

• Locate studies of an issue through objective and
  replicable searches
• Code the studies for salient features
• Code study outcomes on a common scale
• For Kulik and Kulik, this was the effect size
  the difference between the mean scores of two
  groups divided by the standard deviation of the
  control group

17
Meta-Analysis Method

• Use statistical methods to relate study features
  to outcomes
• Quickly gives a broad overview of research in a
  certain field
• May combine apples with oranges rather than
  other apples
• Important to Evidence-Based Medicine

18
Effectiveness of Computer-Based Instruction An
Updated Analysis

• CBI raised final examination scores by 0.30 SD
• Relationship between outcomes and
• Study duration
• Control for instructor effects
• Publication source
• CBI especially effective when duration of
  treatment is four weeks or less.
• Hawthorne effect

19
Effectiveness of Computer-Based Instruction An
Updated Analysis

• Students develop more positive attitudes toward
  computers when they receive help from them in
  school.
• Computers do not have positive effects in every
  area in which they were studied.

20
Richard Clarks Critique of Kulik Kulik

• In over 50 of studies, obvious failures to
  control amount of instruction (bad)
• In over 40 of studies, the same teacher taught
  both CBI and control groups (good)
• Instructional method was controlled for in only
  half of the 30 studies analyzed
• In only 2 of the 15 studies where instructional
  methods were controlled were there significant
  differences favoring CBI
• Meager but compelling evidence for the John Henry
  Effect (compensatory rivalry)

21
Whats Wrong with Media Comparison Studies?
Daniel Surry David Ensminger

• Why do so many people think media comparison
  studies are valuable to our field?
• Prevalent
• Very simple conceptuallyeasy to see the
  variables
• Easy to set up and runespecially if you are
  comparing to the face to face classroom which is
  already set up
• Every time you have a new medium, you have a
  ready made study.
• Provide quick, usable information to answer media
  questions.People are grasping at straws.
• At the core of the field

22
What are the Alternatives?

• Two types of studies
• Media Researchconcerned with effectiveness,
  development and improvement of different media
• Non-media research is concerned with everything
  else---design models, implementation strategies,
  and return on investmentbeyond scope of article.

23
Media Research

• Need to design studies that use the media
  attribute as the independent variable
• For example, explore CAI and vary one of its
  attributes, such as learner control.
• Need to consider learner aptitude
• Similar to Snows Aptitude-Treatment Interactions

24
Aptitude-Treatment Interaction

• Lee Cronbach Richard Snow
• Principles
• Aptitudes and instructional treatments interact
  in complex patterns and are influenced by task
  and situation variables.
• Highly structured instructional environments tend
  to be most successful with students of lower
  ability conversely, low structure environments
  may result in better learning for high ability
  students.
• Anxious or conforming students tend to learn
  better in highly structured instructional
  environments non-anxious or independent students
  tend to prefer low structure.

25
Media Comparison Studies

• In conclusion, researchers in our field should
  avoid being seduced by the simplistic logic of
  media comparison studies. As a field, we should
  force ourselves to look deeper into the important
  issues of instructional method and learner
  characteristics, push ourselves to use a variety
  of experimental, quasi-experimental and
  qualitative research designs, and challenge
  ourselves to conduct methodologically sound
  research.

26
Media Comparison meets the No Significant
Difference Phenomenon

• What is this all about?
• No Significant Difference Web Site
• http//www.nosignificantdifference.org/
• No Comparison Distance Education Finds a New Use
  for No Significant DifferenceLockee, Burton,
  CrossETRD, Vol 47, 3. 1999

27
No Comparison Basic Tenets

• DL Researchers believe that distance learning is
  as rigorous as face to face
• Research is often done by program developers who
  are not basing their research on robust theory,
  good literature review and strong measurement
  techniques
• Difficult to compare DL students (who are
  non-traditional students) to their face to face
  counterparts

28
No Comparison Suggestions

• Longitudinal studies
• Provides a more accurate perspective
• Developmental studies
• A situation in which someone is performing
  instructional design
• The study of the impact of someone elses
  instructional design
• The study of the instructional design process

29
No Comparison Suggestions

• Media Attributes
• Instructional Strategies
• Individual Learner Characteristics