Evaluation Research - PowerPoint PPT Presentation

About This Presentation
Title:

Evaluation Research

Description:

Trochim, W.M. (2006) Research methods ... Studies 43 Professionalism Studies 42 Extension Studies 38 Agriculture Mechanics/Safety Studies Conclusions of ... – PowerPoint PPT presentation

Number of Views:211
Avg rating:3.0/5.0
Slides: 43
Provided by: Kod64
Category:

less

Transcript and Presenter's Notes

Title: Evaluation Research


1
Evaluation Research
  • Kodi D. Havins
  • AED 615
  • Fall 2006
  • Dr. Franklin

2
Evaluation Research
  • Definitions
  • Evaluation is the systematic assessment of the
    worth or merit of some object
  • Evaluation is the systematic acquisition and
    assessment of information to provide useful
    feedback about some object
  • (Trochim, 2006)

3
Evaluation Research
  • Basically is used to provide feedback on an
    event, organization, program, policy, technology,
    person, activity, etc.
  • All of these are called object(s)
  • It assesses information about the object
  • Used heavily in social sciences and government
    agencies
  • Usually outcomes influence
  • decision-making process

4
Research Designs
  • 4 Major Types
  • Survey Research
  • Case Study
  • Field Experiment
  • Secondary Data Analysis
  • (Garson, n.d.)

5
Procedures
  • 4 Strategies
  • Scientific-Experimental Models
  • Management-Oriented Systems Models
  • Qualitative/Anthropological Models
  • Participant-Oriented Models
  • (Trochim, 2006)

6
1. Scientific-Experimental Models
  • Take values and methods from the sciences
    (especially social sciences)
  • Prioritize on the desirability of impartiality,
    accuracy, objectivity, and validity of the
    information generated
  • (Trochim, 2006)

7
1. Scientific-Experimental Models
  • Examples
  • Tradition of experimental and quasi-experimental
    designs
  • Objectives-based research that comes from
    education
  • Econometrically-oriented perspectives including
    cost-effectiveness and cost-benefit analysis
  • Recent articulation of theory-driven evaluation
  • (Trochim, 2006)

8
2. Management-Oriented Systems
  • 2 of the most common
  • Program Evaluation and Review Technique (PERT)
  • Critical Path Method (CPM)
  • (Trochim, 2006)

9
2. Management-Oriented Systems
  • Examples
  • Framework (Logframe) Model
  • Used widely by U.S. Agency for International
    Development
  • (Trochim, 2006)

10
3. Qualitative/Anthropological Models
  • Emphasis on
  • The importance of observation
  • The need to retain the phenomenological quality
    of the evaluation context
  • The value of subjective human interpretation in
    the evaluation process
  • (Trochim, 2006)

11
3. Qualitative/Anthropological Models
  • Examples
  • The approaches known in evaluation as
    naturalistic or Fourth Generation Evaluation
  • The various qualitative schools
  • Critical Theory and art criticism approaches
  • The Grounded Theory approaches of Glaser and
    Strauss
  • (Trochim, 2006)

12
4. Participant-Oriented Models
  • Emphasize the central importance of the
    evaluation participants
  • (Trochim, 2006)

13
4. Participant-Oriented Models
  • Examples
  • Client-Centered Research
  • Stakeholder Approaches
  • Consumer-Oriented Research
  • (Trochim, 2006)

14
  • So, how do you decide which one to use???

15
Things to keep in mind
  • Great evaluators use ideas from each of the 4
    strategies as they need them
  • Methodologies needed will and should be varied
  • This should not be an easy process
  • There is no simple answer!!!
  • (Trochim, 2006)

16
Types of Evaluations
  • There are many different types of evaluations

17
Types of Evaluations
  • 2 Important Types
  • Formative Evaluation
  • Summative Evaluation
  • (Trochim, 2006)

18
1. Formative Evaluation
  • Purpose
  • To strengthen or improve the object being
    evaluated
  • (Trochim, 2006)
  • In other words, to provide useful feedback to for
    the greater good of the object

19
1. Formative Evaluation
  • Includes several Evaluation Types
  • Needs Assessment
  • Evaluablitiy Assessment
  • Structured Conceptualization
  • Implementation Evaluation
  • Process Evaluation
  • (Trochim, 2006)

20
1. Formative Evaluation
  • Questions generally asked
  • What is the definition and scope of the problem
    or issue, or whats the question?
  • Where is the problem and how big or serious is
    it?
  • How should the program or technology be delivered
    to address the problem?
  • How well is the program or technology delivered?
  • (Trochim, 2006)

21
2. Summative Evaluation
  • Purpose
  • To examine the effects or outcomes of some object
  • (Trochim, 2006)
  • In other words, to review the results/conclusions
    of an object

22
2. Summative Evaluation
  • Can be subdivided into different categories
  • Outcome Evaluations
  • Impact Evaluation
  • Cost-Effectiveness and Cost-Benefit Analysis
  • Secondary Analysis
  • Meta-Analysis
  • (Trochim, 2006)

23
2. Summative Evaluation
  • Questions generally asked
  • What type of evaluation is feasible?
  • What was the effectiveness of the program or
    technology?
  • What is the net impact of the program?
  • (Trochim, 2006)

24
Limitations
  • Limitations of this type of research include the
    level of expertise and knowledge of the expert
    panels
  • If the evaluation is only as good as the
    knowledge of the expert panel

25
  • Journal Article

26
A Review of Subject Matter Topics Researched in
Agricultural and Extension Education
  • By Rama B. Radhakrishna Wenwei Xu

27
A Review of Subject Matter Topics Researched in
Agricultural and Extension Education
  • Retrieved from
  • Journal of Agriculture Education
  • http//pubs.aged.tamu.edu/jae/search/default.asp

28
Purpose of Study
  • The purpose of the study was to examine subject
    matter topics researched in agricultural
    education over a ten year period

29
Objectives
  • The objectives of the study were to
  • Identify subject matter topics researched in
    agricultural extension education in the last
    decade (1986 1996)
  • Categorize subject matter topics published in the
    Journal of Agricultural Education and proceedings
    of the National Agricultural Education Research
    Meeting over a ten year period (1986-1996)

30
How the Evaluation Method was used
  • 2 sources were used to gain data
  • Journal of Agricultural Education
  • Papers Presented at the National Agricultural
    Education Research meetings (NAERM) from 1986-1996

31
How the Evaluation Method was used
  • Resulted in 402 journal articles and 451 papers

32
How the Evaluation Method was used
  • Each paper was given a code number
  • Each was reviewed and categorized into relevant
    subject matter categories
  • 3 Criteria were used
  • Title of Study
  • Central Theme and Focus
  • Findings and conclusions

33
How the Evaluation Method was used
  • 25 subject-matter topics were identified
  • The list of the 25 topics was given to a panel of
    experts for review and validation

34
How the Evaluation Method was used
  • Panel of Experts
  • Asked to comment on
  • The appropriateness of categories
  • Add or identify categories they thought that were
    left out or delete categories they felt did not
    fit
  • Suggest whether some categories could be combined

35
How the Evaluation Method was used
  • Revised list of subject-matter categories
    contained 30 subject matter topics
  • Data was then summarized using frequencies and
    percentages

36
Some of the Findings
  • Top 5 Studies
  • 75 Secondary Agriculture Programs Studies
  • 70 Styles/Theory and Cognition Studies
  • 43 Professionalism Studies
  • 42 Extension Studies
  • 38 Agriculture Mechanics/Safety Studies

37
Conclusions of Study
  • Study provided information on subject matter
    topics investigated by agricultural and extension
    educators, which in turn provides perspectives
    about the research efforts of the agricultural
    and extension education profession.

38
Summary of Evaluation Research Method
  • The evaluation method was used in this study to
    assess what topics were researched in the
    agricultural education field during a 10 year
    period.

39
Summary of Evaluation Research Method
  • The outcomes of this study will provide useful
    feedback to those in the profession

40
References
  • Radhkrishna, R.B. Xu, W. (1997). A review of
    subject
  • matter topics researched in agricultural and
    extension education. Journal of Agricultural
    Education. 38(3), 59- 69.
  • Trochim, W.M. (2006) Research methods knowledge
    base.
  • Drake University. Retrieved from
    http//www.socialresearchmethods.net/kb/intreval.
    htm
  • on 10/26/06.

41
Any Questions???
42
Thank You
Write a Comment
User Comments (0)
About PowerShow.com