Objective Evaluation: A Search for Parity in an Online Geography Certificate Program - PowerPoint PPT Presentation

1 / 32
About This Presentation
Title:

Objective Evaluation: A Search for Parity in an Online Geography Certificate Program

Description:

Objective Evaluation: A Search for Parity in an Online Geography Certificate Program ... Squeaky wheel may get all the grease. Approaches Summary ... – PowerPoint PPT presentation

Number of Views:71
Avg rating:3.0/5.0
Slides: 33
Provided by: scottw61
Category:

less

Transcript and Presenter's Notes

Title: Objective Evaluation: A Search for Parity in an Online Geography Certificate Program


1
Objective Evaluation A Search for Parity in an
Online Geography Certificate Program
  • Scott L. Walker, ScEdD
  • Texas State University-San Marcos
  • walks_at_txstate.edu

2
Distance Education Program Evaluation
  • Implies measurement and judgment
  • Poor measurement leads to reliance upon decision
    makers beliefs or beliefs of others that are
    imposed upon the decision maker

3
Gen 5 Distance Teaching/Gen 1 Program Evaluation
  • Distance education program evaluation remains
    less than rigorous
  • Measurements typically fall into three
    categories
  • Student outcomes
  • Attitudes
  • Satisfaction
  • Mostly captured via anecdotal observation or
    low-rigor instruments

4
This presentation is about
  • A rapid, multi-modal evaluation of an online
    geographic information systems (GIS) certificate
    program

5
Program evaluation One way to look at it
  • Overall purpose is to answer questions and
    address issues raised by stakeholders
  • Evaluation combines techniques, proficiencies,
    and sensitivities (politics) required to
    establish
  • If a service is needed
  • If it is conducted as planned
  • If it actually works

6
Program evaluation One way to look at it
  • Embedded within the overall purposes from the
    previous slide are more specific purposes
  • Evaluation is used to
  • Determine accountability and effectiveness
  • ID weaknesses so improvements can be made
  • Gather effectiveness evidence for doubters
  • Provide info for renewal

7
Approaches to Evaluation
  • Objectives-oriented
  • Consumer-oriented
  • Management-oriented
  • Expertise-oriented
  • Adversary-oriented
  • Participant-oriented

8
Objectives-Oriented
  • Are program objectives being met?
  • Poorly-defined objectives
  • Unintended outcomes
  • Hidden agendas
  • Mis-measurement

9
Consumer-Oriented
  • Considers students as consumers
  • Typically summative in nature
  • Does not take into consideration student
    differences (i.e. one size program fits all)

10
Management-Oriented
  • Used to allocate funding
  • Evaluation conducted by those involved in the
    program promote the status quo

11
Expertise-Oriented
  • Program accreditation evaluation
  • Content experts look at program curriculum and
    administration
  • Content expert ? distance education expert

12
Adversary-Oriented
  • Tries to get adversary buy-in
  • Attempts to address adversarys issues

13
Participant-Oriented
  • Qualitative and involves all stakeholders
  • Squeaky wheel may get all the grease

14
Approaches Summary
  • Seldom are these 6 approaches used exclusively
  • Little consensus on which are best
  • Often mixed with course evaluation

15
Measures Actually Used
  • Student grades are the most prominent measure
    second only to technology used.
  • Other measures typically include
  • Class size
  • Target population
  • Student characteristics
  • Admin issues
  • Distance ed vs. face-to-face
  • Requirements of students
  • Requirements of faculty

16
This Evaluation-Background
  • Four-year, doctoral granting, regional university
  • Program consisting of four, 4-semester hour
    online courses
  • Undergraduate level
  • Offers a certificate (not a degree, but courses
    are part of a degree and can be used for
    attaining one)

17
This Evaluation-Background
  • Instructors Two doctoral students with strong
    content background
  • Evaluator Strong content background 10-yrs.
    experience developing, evaluating, teaching, and
    participating in distance ed (eliminates the
    expert-oriented approach problems)

18
This Evaluation-Approaches, Purposes
  • Approaches Combined Management-, Expertise-,
    Participant-Oriented
  • Purposes
  • ID weaknesses to target improvement areas,
  • Ensure parity with equivalent face-to-face
    classes
  • Provide info for renewal

19
This Evaluation-Strategies
  • Determine psychosocial learning environment
  • Content analysis
  • Cost comparison
  • Academic outcomes
  • Retention comparison
  • Benchmarking

20
Psychosocial Learning Environment
  • a.k.a. class climate
  • Used DELES (validated online instrument)

21
Psychosocial Learning Environment
22
Content Analysis
  • Investigates parity between online and
    face-to-face (f2f) sections
  • Considered only one classes content

23
Content Analysis
  • Classroom Content
  • Data input
  • Evaluation quality data
  • Modeling
  • Remote sensing
  • Digital elevation models (DEM)
  • Visualizations
  • Skill Content
  • Personal Web page development
  • Data acquisition
  • Geocoding
  • Spatial analyst
  • Network analyst

24
Cost Comparison
  • Done simply, Considered only personnel costs of
    instructors/lab assistants

25
Academic Outcomes(Grade Distribution Trend)
  • Used only because it is one of the few
    commonalities
  • Aggregated for all students over 3 years

26
Academic Outcomes
27
Academic Outcomes
28
Retention
29
Benchmarking
  • Considers this program in relation to similar
    programs regarding required semester hours and
    delivery mode (online, hybrid, f2f)

30
Benchmarking
  • Mean required semester hours of 25 other GIS
    certificate programs (distance, f2f, hybrid) 19
  • This program 16
  • Mean online GIS certificate programs 16
  • 20 of other university GIS certificate programs
    are online
  • 16 of other programs are offered in some hybrid
    of online and face-to-face
  • 36 not online

31
Summary of Evaluation
  • Presented as SWOT (includes additional
    observation notations)
  • Provides opportunity for decision makers to make
    data-informed decisions from multiple, non-biased
    perspectives

32
Contact Info
  • Scott L. Walker, ScEdD
  • Texas State University-San Marcos
  • http//uweb.txstate.edu/sw36/
  • walks_at_txstate.edu
Write a Comment
User Comments (0)
About PowerShow.com