Using this Resource - PowerPoint PPT Presentation

Loading...

PPT – Using this Resource PowerPoint presentation | free to view - id: 1e5601-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Using this Resource

Description:

Evaluations are used to... Demonstrate accountability. Fulfill ... from Victoria Bernhardt) ... Instead, it is used to identify themes or ... – PowerPoint PPT presentation

Number of Views:32
Avg rating:3.0/5.0
Slides: 24
Provided by: gary71
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using this Resource


1
Using this Resource
  • Click the left mouse button or your space bar to
    progress to the next slide
  • A printer-friendly version of this presentation
    will be available at the conclusion
  • Please note The content of this presentation
    most closely applies to K-12 education in
    Pennsylvania. Other organizations and
    disciplines may have different explanations for
    some concepts.

2
Evaluation 101
  • Yolanda Yugar
  • Leslie Kirby
  • Allegheny Intermediate Unit 3

3
Objectives
  • Participants will
  • Gain a basic understanding of evaluation.
  • Understand the differences between evaluation,
    monitoring, research.
  • Learn about evaluation roles (local, state,
    national).

4
What is Evaluation?
  • Evaluation is the process by which organizations
    examine the implementation and outcomes of an
    intervention to determine whether or not (and to
    what extent) the intervention contributed to a
    change.

5
Understanding the Differences
  • Monitoring
  • Checklist, Review of Documentation
  • Required by Funding Agency
  • Separate from the Evaluation
  • Research
  • Uses research methods such as control group,
    well-matched comparison group, quasi-experimental
    design to determine effectiveness

6
Caution The Notion of Causation
  • It is common to think about a program as causing
    a change. However, it is important to realize
    that programs do not occur in a vacuum. A single
    intervention usually cannot be the sole cause of
    an effect when dealing with people. Other
    programs, interventions, personal situations, and
    even chance contribute to the outcomes discovered
    during evaluation.

7
Evaluations are used to…
  • Demonstrate accountability
  • Fulfill reporting requirements
  • Assess needs
  • Improve programs
  • Determine relationship between cost and benefit
  • Determine feasibility and replicability

8
Types of Evaluation
  • Program implementation To what extent was the
    program implemented as designed?
  • Program outcomes To what extent has the
    initiative influenced behaviors or practices?
  • Program impact To what extent has the
    initiative influenced student achievement?

9
Formative Evaluation
  • occurs while the intervention is in progress
    (usually during the developmental phase)
  • allows management to improve program for current
    term
  • concentrates on implementation and feedback from
    participants and/or program staff

10
Summative Evaluation
  • compilation or summary of what occurred during
    the intervention or a particular term, the
    results of the intervention, and whether goals
    were achieved
  • assesses overall impact

11
Mixed Method Approach
  • Most evaluations use a combination of these forms
    of evaluation to gain a clear understanding of
    the program in order to give possible outcomes a
    context.
  • Using more methods also allows the evaluator to
    examine a single idea or portion using more
    sources of data to strengthen interpretation
    (triangulation).

12
Types of Data
  • Participant demographics
  • Participant achievement
  • Processes
  • Perceptions
  • (adapted from Victoria Bernhardt)

13
Quantitative Data
  • Quantitative data is data that can be counted or
    measured.
  • Includes
  • Assessment/test (achievement) data
  • Participation and attendance rates
  • Forced-choice survey questions (i.e. select
    answer from a list)

14
Qualitative Data
  • Qualitative data is data that cannot be easily
    counted. Instead, it is used to identify themes
    or trends.
  • Includes
  • Focus group responses and Interviews
  • Open-ended survey questions
  • Portfolios or compilations of materials
  • Observations

15
Common Evaluator Tasks
  • Help staff understand evaluation and how it can
    help a program plan and implement more effective
    improvement activities and make corrections in
    those improvement efforts when they are needed.
  • Develop or help to develop the overall plan for
    evaluating the effort.
  • Involve staff in the implementation of the plan
    and, if necessary, train them to carry out
    specific evaluation tasks.
  • Help staff use specific data gathering
    methods/instruments in a reliable way.

16
Common Evaluator Tasks
  • Build capacity to collect, store, analyze, and
    use the data to inform decision-making.
  • Help staff interpret data collected and make
    decisions consistent with the results of specific
    analyses.
  • Write reports that facilitate local use of the
    data collected in decision-making and that
    provide information.
  • Serve as liaison.
  • Help the local effort to achieve its purposes
    through the design and conduct of the evaluation.

17
Selecting an Evaluator
  • You should select an evaluator who
  • Reflects the needed characteristics or
    requirements
  • Understands the organization he/she is evaluating
  • Is willing to work collaboratively to identify or
    develop an evaluation that meets the requirements
    of the initiative and the needs of the
    organization

18
Characteristics of an Evaluator
  • Understands confidentiality of student data.
  • Ability to develop or find appropriate assessment
    instruments.
  • Capacity to collect, manage, analyze, and
    interpret data and produce written reports based
    on that data.
  • Capacity to guide districts in using data for
    decision-making.
  • Understands of state and local needs.

19
Where would I find an evaluator?
  • University or college
  • Intermediate Unit
  • Internal staff with specialized skills
  • Independent contractor
  • For-profit evaluation company

Your local evaluator may be one individual, or a
group of individuals, depending on the
organizations and projects needs.
20
How do I get started?
  • The program and evaluator need to negotiate an
    agreement that sets forth the actual role and
    specific tasks for which the local evaluator will
    be responsible. The agreement would delineate
    how the evaluator will be compensated for his/her
    services.
  • Rule of thumb for services is 8-10 of the total
    project budget depending on the scope of services.

21
Establishing an Evaluation Team
  • The team would include representation from
  • Organization staff
  • Program implementers
  • Evaluator(s)
  • Other stakeholders (community partners, funding
    source, district support services, etc.)

22
Resources
  • Bernhardt, V. (1998) Data Analysis for
    Comprehensive School Improvement
  • Brainard, E. (1996) A Hands-on Guide to School
    Program Evaluation
  • Wahlstrom, D. (2002) Using Data to Improve
    Student Achievement

23
Contact
  • Allegheny Intermediate Unit
  • 475 East Waterfront Drive
  • Homestead, PA 15120
  • (412) 394-5801
  • www.aiu3.net/GEDT.aspx
  • www.aiu3.net/evaluations.aspx
About PowerShow.com