Evaluation of On-Line Learning on Campus http://moby.ucdavis.edu/HRM/IEEE00.htm http://moby.ucdavis.edu/HRM/IEEE00.ppt

1
Evaluation of On-Line Learningon
Campushttp//moby.ucdavis.edu/HRM/IEEE00.htm
http//moby.ucdavis.edu/HRM/IEEE00.ppt
Harry R. MatthewsProfessor and
DirectorInstructional Technology Digital Media
CenterUniversity of California at Davis
2
Authors

• Harry R. Matthews, Professor
• Director, Instructional Technology Digital
  Media Center
• Barbara Sommer
• Teaching Resources Center
• Michael Maher
• Professor, Graduate School of Management
• Curt Acredolo
• Research Professor, Human Community Development
• Arnold Ho, Undergraduate in Psychology
• Richard Falk, Professor.

3
Hypothesis

• Delivery of course content on-line can
• be cost-effective
• improve learning
• be acceptable to students
• be acceptable to faculty.

4
Why?

• Need to accommodate a rapidly growing student
  population.

5
The Strategy

• Create on-line content for
• 10
• large
• undergraduate
• general education courses
• Students choose the on-line or in-person content
• Compare
• costs
• learning
• acceptability.

6
Course names

• Food Science Mythology
• Agricultural Labor
• General Biology
• Environmental Law
• Introduction to Computing
• Computers in Agriculture
• Anthropology
• Statistics
• Introduction to Wine-making
• Asian Art History.

7
Some Variables

• Students choose on-line course
• Appropriate for our question
• Variable use of on-line/in-person content
• Treat as a continuous variable
• Variable on-line presentation
• From mainly text to rich multiple media
• Costs depend on prior materials.

8
Developing On-Line Content one strategy

• Videotape PowerPoint-based lectures
• Digitize and Transcribe sound (voice)
• Convert PowerPoint to JPEG or Flash
• Use Flash Generator major advance
• Make small sound files aligned with the slide
  transitions
• Bookmark the text to correspond with the slide
  transitions
• Assemble in database.

9
Results from the pilot studies

• 5 year study of BCM 410A
• lectures offered in-person in years 1, 2, and 5
• on-line in years 3 and 4
• Aim was to improve higher-order thinking skills
• shift instructor contact-time from lecture to
  discussion sections.

10

• Improved grades in on-line classes
• 2126 A grades during in-person lecture years
• 3555 A grades in on-line years.

11

• Student views of on-line content
• textual analysis of a free-response
  questionnaire.

12
Value for studying virtual lectures
13
Value for studying textbook
14
Results from the pilot studies

• BIS 10 General Biology
• Exam data showed no significant difference in
  student learning between the in-person and
  on-line course
• BIS 10 was mainly text-based no change in
  learning
• BCM410A was multiple media with integrated tools
  improvement in learning
• Is difference in learning, if significant, due to
  the nature of the on-line content?
• What about the quality of the learning?

15
Evaluation of Learning

• Quantitative
• test scores
• Qualitative
• Student perception
• Analysis of tests
• assign questions to Blooms taxonomy of levels of
  learning
• Determine learning style
• Kolb form.

16
Cost Analysis

• Pre-requisite for scalability
• Differential Cost Analysis
• Faculty time
• Assistance for faculty
• Infrastructure (buildings, network)
• Student support
• Costs associated with impacted courses.

17
Cost Analysis

• Major component of cost is instructor time
• Direct time e.g., planning and delivering the
  course
• Indirect time e.g. attending seminars.

18
Instructor time sheet Web form completed
daily Email reminder

• http//moby.ucdavis.edu/Mellon/timesheet.cfm?Name
  Falk

19
Conclusion

• The Jury is IN. On-line learning works!
  David Brown, Jan. 2000
• Does the use of multiple media types affect
  learning? In what situations?
• Is on-line learning cost-effective? Under what
  circumstances?
• Where and to what extent should UC Davis use
  on-line content delivery?

20
Evaluation of On-Line Learning on
Campushttp//moby.ucdavis.edu/HRM/IEEE00.htm
http//moby.ucdavis.edu/HRM/IEEE00.ppt