Tools for Creating a Culture of Assessment The CIPP Model and UtilizationFocused Evaluation

1
Tools for Creating a Culture of AssessmentThe
CIPP Model and Utilization-Focused Evaluation

• Yvonne Belanger Duke University
• Library Assessment Conference
• September 25-27 2006
• Charlottesville VA

2
Key Questions for Libraries

• How can we build a culture of evaluation so that
  many people contribute to evaluation
• How can we provide a context for evaluation
  strategies and results
• How can we conduct evaluation that helps with
  decision making

3
Culture of Assessment

• organizational environment in which decisions
  are based on facts research and analysis and
  where services are planned and delivered in ways
  that maximize positive outcomes and impacts for
  customers and stakeholders.

Lakos et al 1998-2002
4
Barriers to a Culture of Assessment

• Lack of evaluative thinking (at all levels)
• Lack of engagement in evaluation
• Pseudoevaluations (Stufflebeam 1999)
• Promote a positive or negative view of a program
  irrespective of its actual merit and worth

5
Building evaluative thinking CIPP Model

• Stufflebeams CIPP Model - Context Input
  Process and Product evaluation
• Focus decision-making
• Purpose facilitate rational and continuing
  decision-making particularly for programs and
  services with long-term goals

6
Details of the CIPP Model

• CIPP
• Context Environment Needs
• Input Strategies Resources
• Process Monitoring implementation
• Product Outcomes - both quality and significance
• More information at www.wmich.edu/evalctr/pubs

7
CIPP approach recognizes

• All politics are local offers a tailored
  evaluation approach designed to answer locally
  interesting useful questions emphasis is on
  credibility and usefulness rather than
  generalizability to other places times audiences

8
CIPP View of Institutionalized Evaluation
Stufflebeam OPEN 2003
9
Advantages of the CIPP Model

• Adapts well to carrying out evaluations on any
  scale (projects programs organizations)
• An organizing framework not a lockstep linear
  process
• Sensitive to needs of decision makers
• Systems approach encourages a systems view of
  projects and programs

10
Building evaluative thinking and engagement
Utilization-focused evaluation approach

• Taking a utilization-focused approach means
  asking
• Why is this evaluation being undertaken
• What decisions need to be made with the results
• Who will be most affected by those decisions
• How can we engage those people in the entire
  evaluation process

11
Utilization-focused evaluation

• Premise by engaging stakeholders in the entire
  evaluation process from design to implementation
  of recommendations
• Evaluation addresses questions of greatest
  importance to those in a position to directly
  make use of its findings
• Reduces the cultural barriers that can inhibit
  use of results by increasing transparency
  empowering stakeholders

12
Another advantage of the Utilization-focused
approach

• Process Use benefits
• First described by Patton - ways in which being
  engaged in the processes of evaluation can be
  useful quite apart from the findings that may
  emerge from these processes
• Four types of Process Use
• 1. Enhancing shared understandings especially
  about results
• 2. Supporting and reinforcing the object of the
  evaluation through intervention-oriented
  evaluation
• 3. Increasing participants engagement sense of
  ownership
• 4. Organizational development

Patton 1997 pp. 88-91
13
Process Use Culture of Assessment

• Increased capacity to make use of evaluation
  findings
• Know how to use evaluation information
  producing better evaluation users in the
  organization who can effectively weigh evidence
  consider contradictions and inconsistencies
  articulate values and examine assumptions

Patton 2004 On Evaluation Use Evaluative
Thinking and Process Use
14
Example

• Evaluation of the Duke iPod experiment Duke
  Digital Initiative

15
Summary

• Foster a culture of assessment by
• Adopting frameworks that support decision-making
• Engaging staff as stakeholders in the entire
  process of evaluation from design to
  implementation of recommendations
• Leverage the opportunity of Process Use to
  develop staff and make them more saavy evaluation
  consumers

16
Final Thoughts

• evaluations most important purpose is not to
  prove but to improve.
• Daniel Stufflebeam
• (CIPP Model)
• Research is aimed at truth. Evaluation is aimed
  at action.
• Michael Quinn Patton
• (Utilization-focused Evaluation)

17
Thank You!

• Yvonne BelangerHead Program Evaluation
• Academic Technology Instructional Services
• Perkins Library
• Duke University
• yvonne.belanger_at_duke.edu

18
References

• Stufflebeam D. (1999). Foundational models for
  21st century program evaluation.
• Stufflebeam D. (2003). The CIPP Model for
  Evaluation An update a review of the models
  development a checklist to guide implementation.
  Paper read at Oregon Program Evaluators Network
  Conference at Portland OR. http//www.wmich.edu/
  evalctr/pubs/CIPP-ModelOregon10-03.pdf
• Patton M. Q. (2004). On evaluation use
  Evaluative thinking and process use. The
  Evaluation Exchange IX(4).
• Patton M. Q. 1997. Utilization-focused
  evaluation The new century text (3rd ed.).
  Thousand Oaks CA Sage.