Summative Evaluation - PowerPoint PPT Presentation

About This Presentation
Title:

Summative Evaluation

Description:

Title: Summative Evaluation Author: Barry O. Williams Last modified by: John Hollenbeck Created Date: 11/14/1996 6:00:19 PM Document presentation format – PowerPoint PPT presentation

Number of Views:139
Avg rating:3.0/5.0
Slides: 25
Provided by: Barr2169
Category:

less

Transcript and Presenter's Notes

Title: Summative Evaluation


1
Summative Evaluation
  • The Evaluation after implementation

2
Involves _______ data
  • Collecting
  • Analyzing
  • Summarizing

3
For the purpose of
  • Giving decision makers information on the
    effectiveness and efficiency of instruction

4
Effectiveness of Content
  • Instruction solve the problem?
  • Criterion created prior to evaluation?
  • Was the criterion established in conjunction
    withthe needs assessment?

5
Specifically
  • Did learners achieve the objectives?
  • Learners feeling about instruction?
  • What were the costs?
  • How much time did it take?
  • Was instruction implemented as designed?
  • What unexpected outcomes?

6
Alternative Approaches to Summative Evaluation
  • Objectivism
  • Subjectivism

7
Objectivism
  • Based on empiricism
  • Answering questions on the bases of observed data
  • Goal based and replicable, uses the scientific
    method

8
Subjectivism
  • Employs expert judgment
  • Includes qualitative methods
  • observation and interviews
  • evaluate content
  • Goal Free
  • evaluators havent a clue about the goals

9
Objectivism (limitations)
  • Examine only a limited number of factors
  • May miss critical effects

10
Subjectivism (limitations)
  • Are not replicable
  • Biased by idiosyncratic experiences,
    perspectives, or the people who do the evaluation
  • May miss critical effects

11
Designer Role in Summative Evaluation?
  • Somewhat controversial

12
Timing of Summative Evaluation?
  • Not in the first cycle

13
Summary Diagram
  • Formative
  • Design Reviews
  • Expert Reviews
  • One-to-one Eval.
  • Small Group Eval.
  • Field Trials
  • Ongoing Eval.
  • Summative
  • Determine Goals of the Evaluation
  • Select Orientation
  • Select Design
  • Design or Select Evaluation Measures
  • Collect Data
  • Analyze Data
  • Report Results

14
Goals of the Evaluation
  • What decisions must be made?
  • What are the best questions?
  • How practical is it to gather data?
  • Who wants the answer to a question?
  • How much uncertainty?

15
Orientation of Evaluation
  • Goal-based or goal-free
  • A middle ground?
  • Quantitative or qualitative appropriate?
  • Experimental or naturalistic approach?

16
Select Design of Evaluation
  • Describes what data to collect
  • When the data will be collected
  • And under what conditions
  • Issues to consider
  • How much confidence must we have that the
    instruction caused the learning? (internal
    validity)
  • How important is the generalizability? (external
    validity)
  • How much control do we have over the
    instructional situation?

17
Design or Select Evaluation Measures
  • Payoff outcomes
  • Is the problem solved?
  • Costs avoided
  • Increased outputs
  • Improved quality
  • Improved efficiency

18
Design or Select Evaluation Measures (2)
  • Learning Outcomes
  • Use instrument youve already developed for the
    summative evaluation
  • But measure the entire program

19
Design or Select Evaluation Measures (3)
  • Attitudes
  • Rarely the primary payoff goals
  • Ask about learner attitudes toward
  • learning
  • instructional materials
  • subject matter
  • Indices of appeal
  • attention, likeableness, interest, relevance,
    familiarity, credibility, acceptability, and
    excitement

20
Design or Select Evaluation Measures (4)
  • Level of Implementation
  • degree to which the instruction was implemented
  • Costs
  • Cost-feasibility
  • Cost-effectiveness

21
Alternative Designs
  • Instruction then posttest
  • Pretest then instruction then posttest

22
The Report
  • Summary
  • Background
  • Needs assessment, audience, context, program
    description
  • Description of evaluation study
  • Purpose of evaluation, evaluation of the design,
    outcomes measured, implementation measures,
    cost-effectiveness info., analysis of
    unintentional outcomes

23
The Report (continued)
  • Results
  • Outcomes, implementation, cost-effectiveness
    info., unintentional outcomes
  • Discussion
  • causal relationship between program results
  • Limitation of study
  • Conclusion Recommendations

24
Summary
  • Summative evaluation is after implementation
  • Limitations of subjective and objective
    evaluation
  • What to include in the report
Write a Comment
User Comments (0)
About PowerShow.com