Research methods in clinical psychology: An introduction for students and practitioners Chris Barker

1
Research methods in clinical psychologyAn
introduction for students and practitionersChris
Barker, Nancy Pistrang, and Robert Elliott

• CHAPTER 11
• Evaluation research

2
Outline

• Background concepts
• Preparing to evaluate a service
• Monitoring service delivery
• Outcome evaluation

3
What is evaluation?

• Evaluation is applied research that aims to
  assess the worth of a service, often judging it
  against specified goals
• Two central questions
• What is the service trying to do?
• How will you know if it has done it?

4
Characteristics of evaluation

• For local consumption only
• Aid to decision making
• utilisation-focused
• Takes place in an action setting
• Participants are service users
• Tight timetable

5
Some terminology

• Audit the audit cycle
• Quality assurance
• Clinical governance
• Formative versus summative evaluation
• Stakeholders
• Structure, process, outcome (Donabedian)

6
The audit cycle

• Set standards

Observe practice
Make changes
Compare
7
Socio-political issues

• Evaluation is threatening
• Being scrutinised
• Time consuming
• May be used against you
• May not validly assess your work
• External versus in-house evaluations
• Different stakeholders may have different agendas

8
Preparing to evaluate a service

• Aims and objectives
• Needs assessment
• Service design
• (Rossi, Freeman Lipsey, 1999)

9
Aims and objectives

• Aims global statements of desired outcomes
• Objectives specific goals
• ideally occur within a specific time period
• detail what the service will do to achieve its
  aims
• indicate whether or not the aims will have been
  met
• are clear, simple and, if possible, measurable

10
Needs assessment

• specify the target problem and the target
  population
• estimate the extent of the problem
• assess the need for the service
• need (professionally determined)
• demand (client determined)
• supply (availability)

11
Service design

• Impact model
• links problem cause, service action, expected
  outcomes
• Delivery system design
• operational plan

12
Process evaluationmonitoring service delivery

• Who does what to whom?
• Coverage
• Who is getting the service?
• (consider issues of bias)
• Implementation
• What service is being given?

13
Outcome evaluationWhat was the impact of the
service?

• Efficacy
• evaluation under controlled experimental
  conditions
• Effectiveness
• evaluation of the service as delivered in
  practice (see Seligman, 1995)
• Client satisfaction surveys
• Cost-benefit analysis