Title: An International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT): Research Framework and Protocols Robert Elliott University of Toledo
1An International Project on the Effectiveness of
Psychotherapy and Psychotherapy Training
(IPEPPT) Research Framework and Protocols
Robert ElliottUniversity of Toledo
2Research-Practice Gap in Era of Evidence-Based
Mental Health
- Numerous contemporary attempts to link research
practice in psychotherapy - Top-down solutions
- Empirically-supported treatments
- Evidence-Based Practice
- Based on
- Randomized Clinical Trials research model
- Therapist-as-research-consumer model
- Results have been mixed
3Research-Practice Integration as a Two-way,
Dialectic Process
- Success is more likely if we add a more
integrative, bottom-up strategy - Building on Mental Health Services/ Therapy
Effectiveness paradigm - Existing RCT research makes space for
grass-roots-based research in real world practice
and training settings - Practice-based Evidence
4Example Practitioner Research Networks (PRNs)
- USA Pennsylvania (Ragusea, Borkovec, Castonguay)
- UK National Health Service CORE research team
(Barkham, Evans et al.) - Latest trend Practice-based research in training
clinics and centers (e.g., Castonguay et al.)
5Practice-Based Therapy Research in Training Sites
- Training site research movement USA, Europe
- Research on psychotherapy process/outcome is
essential for understanding and improving
psychotherapy practice in all orientations - Being able to use and carry out research is an
important aspect of therapist competence - Best way to learn therapy research methods
- Do research during basic therapy training
- Primary professional socialization process
- Create habits that carry over into later practice
6Principles for Practice-based Research
- Make research relevant to actual practice of
therapy - Use methods that support therapy rather than
interfere with it - Actively and continuously involve therapists in
selection of research questions and methods - Include inexpensive and easy-to-use instruments
of key elements (therapeutic alliance, client
problem severity) - Encourage variety of research methods
(qualitative quantitative group single-case)
- Create research networks of training sites using
similar, pan-theoretical instruments
7International Project on the Effectiveness of
Psychotherapy and Psychotherapy Training (IPEPPT)
- Formally initiated, June 2004, by
- Italian Coordinamento Nazionale Scuole di
Psicoterapia (CNSP gt5,000 therapists) - Italian Federation of Psychotherapy Associations
(FIAP 21 psychotherapy associations gt10,000
therapists) - General Goal To improve psychotherapy and
psychotherapy training in a broad range of
theoretical approaches by encouraging systematic
research in therapy training institutes and
university-based training clinics.
8IPEPPT General Scientific Steering Committee
- Robert Elliott, Scientific Director (University
of Toledo-USA) - Alberto Zucconi, Coordinator (University of
Siena-Italy) - David Orlinsky (University of Chicago-USA)
- Franz Caspar (University of Freiburg)
- Louis Castonguay (Pennsylvania State
University-USA) - Glenys Parry (University of Sheffield-UK)
- Bernhard Strauss (Friedrich Schiller University
Jena-Germany)
9IPEPPT Current Status
- Still in formation stage
- Not a single study
- The Project Promoting practice-based research
in Europe, North American and elsewhere - Finding partners
- Creating/finding tools
- E.g., conceptual/organizing concepts
10IPEPPT Specific Objectives
- 1. To construct a list of agreed-upon general
pantheoretical recommendations for evaluating - Key aspects of therapy, especially in training
centers - Key aspects of therapy training outcome
- Not a Core Battery
- 2. To facilitate the development of specific
treatment and training outcome protocols for
particular - Therapy approaches (e.g., Systemic therapy)
- Client populations (e.g., people living with
schizophrenia) - Linguistic/national groups (e.g., Italy)
- 3. To facilitate national/international
collaborations
11IPEPPT Draft Research Framework
- Such a project requires a guiding conceptual
framework for determining what to measure and how
to measure it - Work-in-progress
- 8 measurement domains
- 4 Research themes
- 2 Levels (Star design)
12Framework Eight Therapy Measurement Domains,
with examples of key concepts
Research Theme A. General/ Pantheoretical B. Treatment- Specific
I. Therapy Outcome e.g., general problem severity e.g., theory-based dysfunctional processes
II. Therapy Process e.g., therapeutic alliance e.g., therapist techniques
III. Client/Ther-apist Background e.g., demographics e.g., preference for type of therapy
IV. Training Outcome e.g., productive vs. unproductive practice pattern e.g., therapist skill development
13Structure (1) Star Design
- Main body of the star General outcome/ process
protocol - Shared by all orientations (General/
Pantheoretical) - Provides common metric
- Star rays Specialized protocols for different
therapy approaches and different countries
(Treatment/Population/Language Specific)
14Star Design for Sample Concepts within Therapy
Outcome Domain for Studies of Four Different
Therapies
15Structure (2) Nested Priority Lists
- Not a single core battery
- Allow flexibility while encouraging consistency
within across approaches - Three Levels of Priorities
- Measurement domains are prioritized
- Within each measurement domain, key concepts are
ranked by approximate importance - For each concept, available instruments are also
described (researchers prioritize)
16Framework Eight Therapy Measurement Domains,
with examples of key concepts
Research Theme A. General/ Pantheoretical B. Treatment- Specific
I. Therapy Outcome e.g., general problem severity e.g., theory-based dysfunctional processes
II. Therapy Process e.g., general problem severity e.g., therapist techniques
III. Client/Ther-apist Background e.g., demographics e.g., preference for type of therapy
IV. Training Outcome e.g., productive vs. unproductive practice pattern e.g., therapist skill development
17Example General Therapy Outcome Domain
- Key concepts in a possible recommended priority
order (Star) - (1) General problem severity (quantitative)
- Give every 2 sessions to reduce data loss from
drop-out - (2) Interpersonal/relational functioning
- (3) Qualitative perceptions of change
- (4) Individualized problems/goals
- (5) Health care utilization/costs
- (6) Quality of life/life satisfaction/well-being
18Common General Symptom Severity Instruments
19Framework Eight Therapy Measurement Domains,
with examples of key concepts
Research Theme A. General/ Pantheoretical B. Treatment- Specific
I. Therapy Outcome e.g., general problem severity e.g., theory-based dysfunctional processes
II. Therapy Process e.g., therapeutic alliance e.g., therapist techniques
III. Client/Ther-apist Background e.g., demographics e.g., preference for type of therapy
IV. Training Outcome e.g., productive vs. unproductive practice pattern e.g., therapist skill development
20Example General Therapy Process Domain
- Key concepts in possible recommended priority
order - (1) Therapeutic alliance
- (2) Therapist and client response modes
- (3) Perceived helpful aspects of therapy
- (4) Perceived session effectiveness
21(No Transcript)
22Different Levels of Research Protocol are Possible
- I. Minimum Protocol
- II. Systematic Case Study Protocol
- III. Maximum Protocol
- Other Protocols
- IV. General Training Protocols
- V. Specific Research Protocols
23I. A Recommended Minimum Protocol Applications
- Easy to use Limited to one measure from each of
the first three research domains - Can use with own clients
- Provides basic treatment monitoring for
individuals agencies - Other versions are possible (e.g., different
outcome or process measures)
24I. A Recommended Minimum Protocol Elements
- (1) General therapy outcome instrument
- Client problem severity
- Give at odd-numbered sessions (short form)
- (2) General therapy process
- Therapeutic alliance (use short from)
- (3) Client/therapist background measure
- Standard practice
- Client/ therapist demographics
- Client diagnosis, presenting problems
- Type of therapy
25II. Systematic Case Study Protocol Applications
- Use for student case study requirements
- Meets emerging standards for systematic single
case research - New online journal Pragmatic Case Studies in
Psychotherapy (Rutgers University, Editor
Fishman)
26II. Systematic Case Study Protocol Elements
- A. Therapy Outcome
- (1) Weekly/biweekly outcome measure
- (2) At least one other quantitative outcome
measure - (3) Qualitative outcome assessment (e.g.,
post-therapy interview) - B. Therapy Process
- (1) Therapeutic alliance
- (2) Detailed record of therapy (process notes
and/or recordings) - (3) Qualitative perception of helpful aspects
(post-session and/or post-therapy) - C. Client/therapist background
- Client/ therapist demographics client diagnosis,
presenting problem type of therapy
27II. Systematic Case Study Protocol Research
Questions
- (1) Did the client change substantially over the
course of therapy? - (2) If the client changed, did therapy make a
substantial contribution? - (3) What brought about the clients changes?
28II. Systematic Case Study Protocol Emerging
Evidence Standards
- (1) Rich case record, including both quantitative
qualitative data - (2) Replication/convergence across methods
- (3) Critical examination of alternative views
(e.g., Hermeneutic Single Case Efficacy Design,
Elliott, 2002) - Non-change explanations (e.g., measurement error)
- Non-therapy explanations (e.g., extra-therapy
events) - (4) Narrative coherence
- Narrative model of predisposing and process
factors - Use for generalizing to other cases
29III. Maximum Protocol
- Include measures of at least one concept in each
of the eight domains - Appropriate for research centers (e.g., Center
for the Study of Experiential Psychotherapy) - Also consortia of cooperating centers
- Each center measure some variables
30IV. General Training Research Protocols Issues
- Outcomes of therapy training not well understood
- Difficulties
- Lack of agreed-upon measures of therapist
functioning and skill - Must measure therapist change longitudinally over
several years of training - Possible applications
- Use research to improve training
- Meet requirements of accrediting and funding
agencies
31IV. General Training Research Protocols
Promising Concepts
- General therapist facilitative interpersonal
skills (e.g., coping with common difficulties) - Quality of therapist professional involvement and
growth (e.g., Orlinsky Collaborative Research
Network CRN) - Qualitative perceptions of effects and important
aspects of training (e.g., qualitative
interviews) - Change in therapist self concept (e.g., Scilligo,
SASB Introject scales)
32V. Specific Protocols
- Star rays
- Applications For specific theoretical
approaches, client populations, or language
groups - Requires working committee for each group
- Identify relevant therapy outcomes, processes,
background variables (or training outcomes) - Do protocol and measure development research
- Establish virtual communities for exchanging ideas
33V. Specific Protocol Example
- Person-Centered and Experiential Psychotherapy
International Research Group (PCEP-IRG) - Current core members
- University of Toledo (Elliott team)
- Ohio University (Anderson team)
- Katholieke Universiteit Leuven (Leijssen team)
- Universities of Strathclyde Abertay, Scotland
(McLeod, Cooper)
34V. PCEP-IRG Outcome Protocol Promising
Developments
- Center for the Study of Experiential Therapy
Research Protocol (CSEP- 2) - Self--determined problems/goals
- Personal Questionnaire (PQ-10)
- Self-concept (content coherence)
- Qualitative Self-Description interview
- Tennessee Self-Concept Scale 2 (long, short
forms) - Experiential processing
- Toronto Alexithymia Scale (TAS-20)
- Need positive mental health measures,
self-coherence, etc.
35V. Specific Training Research Protocols
- Some Possible Types of Specific Training
Outcomes - Treatment-specific intervention skill
- Case formulation skill
- Therapist personal development (e.g., maturity,
identification with orientation, values)
36Promising New Therapy Research Methods Make this
Work Possible
- Systematic qualitative research methods
- Interpretive single case designs (Fishman,
Elliott) - Using early outcome to identify repair problems
(Lambert Signal alarm methods) - New, powerful psychometric methods (Rasch
analysis/Item Response Theory) - Virtual communities (Community Zero)
37Invitation to Dialogue - 1
- 1) Provide comments and suggestions on the
framework concepts presented here
Robert.Elliott_at_utoledo.edu - 2) Form or join online discussion groups or
virtual communities - Closed sites must apply for membership
- General info www.communityzero.com/ipeppt
- Example www.communityzero.com/pcepirp
- 3) Begin implementing the minimum protocol design
with your own clients and in your own training
setting.
38Invitation to Dialogue - 2
- 4) Convert traditional case presentation training
requirements into systematic case study exercises - 5) Help with translations of key research
instruments - 6) Contribute to psychometric research
- Improve existing instruments
- Equate different instruments for same concepts
- 7) Collaborate with groups with similar interests
to generate data for pooling.