An Evaluation of Mediabased Online Courses and Emerging Course Criteria

1
An Evaluation of Media-based Online Courses and
Emerging Course Criteria

• Lisa Cheney-Steen, Academic Dean, Colorado
  Community Colleges Online
• Rhonda M. Epper, Director of Online Program
  Development, Community College of Denver

2
Monterey Institute for Technology and Education
(MITE)

• Online Course Evaluation Project (OCEP)
• MI-TE is an educational non-profit that promotes
  the development and distribution of high-quality
  online courses

3
Online Course Evaluation Project (OCEP)

• First phase funded by The William C. and Flora
  Hewlett Foundation
• Collaboration with EduTools and WCET
• Identify and evaluate online courses in higher
  education and Advanced Placement?.
• Develop a set of selection criteria to define
  developers and courses to evaluate

4
Online Course Evaluation Project (OCEP)slide
two

• Develop a set of objective categories for
  evaluation of online course and content
• Examine courses to determine if they meet a high
  level of scholarship, scope, instructional design
  and interoperability (technology)
• Includes assessment of the course developers
  development and operational models for ensuring
  course distribution and sustainability

5
Course and Developer Selection Criteria

• Curricular Courses
• Object Architecture
• Development Model
• Content Model
• Multimedia Rich
• Course Investment

6
Developers Surveyed

• Surveyed over 170 developers
• Online educational organizations and virtual
  schools
• Individual educational institutions and programs
• Private/commercial course developers
• Educational publishers
• First phase of evaluation included 15 developers

7
Evaluation of Online AP Courses

• Multiple partners
• Western Consortium for Accelerated Learning
  Opportunities (WCALO), a project of WICHE, funded
  by US Dept of Education.
• WCET (EduTools)
• Monterey Institute (OCEP)

8
Evaluation of Online AP Courses

• WCALOs Goal
• to increase the number of low-income students
  who are enrolling and succeeding in advanced
  placement courses and tests.
• WCALO identified a need to assist school
  administrators teachers in selecting online AP
  courses for their students

9
Evaluation of Online AP Courses

• WCALO and WCETs EduTools are partnering to
  create a pilot EduTools site with reviews of
  online AP courses
• 14 courses reviewed in four subject areas
• Biology, Calculus AB, US History, English Lit.
• 5 course providers/developers
• Wide range of development investment

10
The Evaluation Categories and Process

• The Evaluation Team
• Project Director
• Course Evaluator
• Academic Evaluator
• Technology Evaluator
• Course Developer Review and Comment

11
The Evaluation Categories and Process

• Categories organized into 7 major divisions
• Scope and Scholarship
• User Interface
• Course Features and Production Values
• Assessments and Test Items
• Instructor and Learner
• Technology
• Distribution Model
• Each category is defined and consistent
  guidelines provided for evaluators

12
Course Development Trends

• Course development models
• Faculty-driven, supported, and centralized
• Content development models
• Textbook-centric, content online, multimedia
• Use of multimedia
• Investment models
• Focus on learning object architecture and
  interoperability

13
Course Evaluation Trends

• Content is most important, CMS less important
• Use of multimedia, experiential exercises, and
  game playing as integral component of the
  dissemination of content rather than an ancillary
  feature
• Learning object development and centralized
  storage of content
• Interoperability, greater distribution

14
Next Phase of OCEP

• Undergoing revision of the research design
• Revise evaluation categories per feedback
• More detailed training for evaluators to ensure
  consistency
• Expand developers and courses to be evaluated
• Launch public access to information

15
An Evaluation of Media-based Online Courses and
Emerging Course Criteria

• Lisa Cheney-Steen, Academic Dean
• Colorado Community College Online
• Lisa.Cheney-Steen_at_cccs.edu
• Butch Gemin
• Monterey Institute for Technology and Education
• bgemin_at_montereyinstitute.org
• www.montereyinstitute.org