Running an Online Section of a Large Introductory Course: Lessons from a ThreeYear Pilot

1
Running an Online Section of a Large Introductory
Course Lessons from a Three-Year Pilot

• Diane Gifford-Gonzalez
• Professor, Anthropology, UC Santa Cruz
• Kelly Stack
• Faculty Support Specialist, Faculty Instructional
  Technology Center, UC Santa Cruz
• UC 21st Century Teaching Technologies Past,
  Present and Future
• UC Davis June 20-21, 2008

2
Anthropology 3 Introduction to Archaeology

• 315 students, 7 Teaching Assistants
• mandatory once-weekly discussion-demo sections
• course size limited by funding allocations for
  TAs
• Required for Anthropology major
• UCSC General Education requirement
• Enrollment 2006-2008
• 46 Lower Division
• 54 Upper Division
• 24 Anthropology majors/prospective majors

3
Course Content

• Archaeologists alternate 3-yr rotas with
• shared core curriculum
• common readings
• 1990s Judith Habicht-Mauche developed
  Anthropology 3 Workbook Study Guide
• funded by UCSC Committee on Teaching funds
• DGG revised per slight module content differences

4
Diversity of Majors 2006-2008

• Anthropology majors generally preferred live
  lecture format

5
Emerging Issues in Late 1990s

• High levels of lecture absenteeism (60)
• Core of motivated students in huge hall
• audience of 85 in 400-seat room - no juice

Aside from forcing people who dont want to
attend to do so by taking attendance and counting
it toward their grade, what options exist?
6
Rethinking Teaching Goals

• Teaching Goal 1 Have only those students in the
  lecture space who really want to be there and
  participate. Create more of a small class
  dynamic.

Teaching Goal 2 Accept the reality that some
students could not or will not attend lectures.
Ensure that they nonetheless meet all course
requirements and actually learn something about
archaeology
7
Responding to New Pedagogic Needs

• 2004-2005 JHM initiates web-archiving videos of
  lectures
• web-archiving by Sheryl Martin-Schultz
  Instructional Media Svcs
• WebCT/Blackboard-based quizzes on lecture content

• 2005 DGG obtains CEP okay for 3-year trial of
  live-lecture plus online course sections
• students commingled in weekly, mandatory
  discussion-demonstration sections
• all submit same work, including weekly online
  quizzes
• Online Surveys on content delivery of both
  sections _at_ end of course
• 2006 210 in live lecture, 105 online
• 2007 165 in live lecture, 150 online
• 2008 97 in live lecture, 205 online

8
Practicalities Online Resources

• WebCT/Blackboard CE 4.1 platform
• Syllabus in pdf and html formats
• Lecture content
• Powerpoint lectures in pdf form
• note-taking and slide-by-slide formats
• streaming videos
• 2-3 hours after live lecture
• podcasts
• 1 hour after live lecture
• Instructor, TA contact info, office hours

• Portal to digital versions of videos viewed in
  lecture
• Links to weekly quizzes
• Links to web-based exercises, enrichment resources

9
Online Quiz-Based Assessment

• Weekly online quizzes - 40-hour window, untimed,
  one try only
• Friday 8 am - Saturday midnight
• open-book multiple choice, focus on lecture, film
  content
• 20 questions randomly chosen from a bank of
  28-30 questions
• question order and answer order set to randomize
  on each student attempt
• answer-based feedback indicates correct
  answer with each response
• to motivate weekly processing of lecture
  content

10
Assessing the Pilot

• Mode 1 Student Assessments
• Online Survey optional comments (next slide)
• formal Online Survey question responses (to come)
• comments on content delivery in
  Course/Instructor/TA Evaluations
• faculty emails with students during enrollment
  period
• comments in faculty office hours, drop-in
  sessions for online students, etc.

• Mode 2 Student performance in the two sections,
  as indexed by average course GPAs
• Mode 3 Logistical costs and benefits of mounting
  the course, using alternative forms of content
  delivery

11
Mode 1 Student Feedback

• Online students optional comments were 68
  positive
• online option offers convenience, flexibility
• can take two courses scheduled _at_ same hour
• can fit with work/athletics/family schedule
• can choose section according to individual
  learning mode
• some learned they did not have discipline to keep
  up the hard way
• some opted to start attending live lecture

This was an awesome alternative to live lecture!
It helped me to have a not so stressful quarter,
because I could do it all on my own time at my
own pace.

• Live lecture students were 68 positive
• valued archived digital lectures and PowerPoints,
  to make up missed lectures
• to review modules before quizzes

I found the webcast lectures and slides
extremely helpful in reviewing information. Also,
with the lectures being online, it was really
convenient for sick days--we all have them, and
it is rare that you can go online and just watch
your lecture if you are not feeling well enough
to go onto campus. I really appreciate all of the
hard work put into this course with the webcasts.
Thanks, all!
I definitely always prefer to attend live
lectures, but having recordings of the lecture as
well as notes and slides available as a resource
was awesome. I wish all my classes were like that
so that I could go back and check something again
in case I missed it. I definitely think this
system works, and I love that there was no
final!! That made things much less stressful. And
I found that studying for a quiz each week helped
me learn the material much better than just
cramming it into one test.
12
2006-2008 Online Survey Feedback
13
Mode 2 Student GPA

• Online section GPA averaged 0.12 lower than in
  the live lecture all three years
• this is not a statistically significant
  difference, but
• Is this due to delivery mode? Perhaps not.
• Some Upper Division non-Anthropology majors told
  TAs they did not intend to work for the highest
  grade possible in Anthro 3
• they were devoting their energies to getting a
  high GPA in their majors

• Dr. Julian L. Fernald, Director of Institutional
  Research, Analyst Sirinda Sincharoen, UCSC Office
  of Planning Budget, regressed multiple student
  variables against Anthro 3 GPA for each section
• class level, major, gender
• number of courses and of units attempted
• average GPA for other courses taken same quarter

14
Mode 2 Student GPA Analysis

• Course delivery mode did not predict Anthro 3 GPA
  
• Nor did any other variables, except one
• GPA in other courses attempted the same term
• online section students were marginally less
  successful than live lecture students in all
  their courses, not just in Anthro 3

In summary, the relationship between residual
GPA and course grade was statistically
significant for both on-line (r.301) and live
(r.325) sections. It is important to note that
while they were significant they are on the weak
side. J. Fenald 2 June 2008
15
Mode 3 Cost-Benefits of Delivery

• Podcast is more cost-effective than streaming
  video
• Streaming video needs special rooms with video
  setup, 2-3 Instructional Media staff for camera
  work, sound monitoring, converting videos
• vs
• Podcast data/audio recorded directly on
  instructors or rooms computer, posted directly
  from there to website

16
Mode 3 Other Costs of Course

• Course is a labor-intensive but scalable process
• Instructor, TAs usual work but heightened email
  traffic
• online students unsure of system require initial
  coaching
• as more such courses offered on campus, less help
  is needed
• Plan to offer in such a format for several years
• recoup investment in online quizzes, other LMS
  website setup
• Prior faculty LMS experience is a prerequisite
• Multi-faculty buy-in to course may avoid
  individual fatigue

17
We Also Learned

• On-call technical staff is required for
  trouble-shooting, e.g.
• the lecture rooms PowerPoint slide remotes
  broadcast frequency initially overlapped that of
  the lecturers microphone
• when remote was used to advance PowerPoint
  slides, it cut out the lecture audio transmission
  and recording - creating havoc with sound in
  archived data
• Glitches require quick fixes, because they have
  knock-on effects on hundreds of online students
• course morale rises and falls on the stability of
  the platforms and the quality technical support
• tomorrow afternoon is not good enough

18
Final Remarks

• Anthro 3 size was limited not by intrinsic traits
  of format, but by instructional support levels
• We could readily have taught twice the
  enrollment, but only had funding for 7 TAs
• online format produced no numeric gain over
  traditional format
• in a course with TAs and sections as a vital
  pedagogic link
• We were able to serve a broader range of student
  life situations and learning styles
• while maintaining standards for course mastery