Roles of Common Factors

1
Roles of Common Factors Therapist Effects
in Therapy Outcomes Session 0931
G.S. (Jeb) Brown, Ph.D. Center for Clinical
Informatics
2
Common factors

• The effectiveness of all treatments is due, in
  some part, to factors common to all treatments.
• Contact with a helping, caring professional
  fosters hope and expectancy.
• We have come to accept the potency of placebo
  effects, and insist that the effectiveness bona
  fide treatments exceeds that of placebo
  treatments
• So far, so good. Who can argue with this?

3
Randomized double-blind placebo controlled drug
trials

• Double blind placebo controlled drug studies
  provide an exemplar for estimating the role of
  common factors.
• Traditionally, the drug is interpreted as the
  difference between placebo and the active drug.
• Meta-analysis of multiple studies of
  antidepressants lets us estimate the relative
  importance of common factors (placebo effects)
  versus drug effects.

4
Meta-analyses and placebo

• Meta-analysis involves the use of a statistical
  techniques to combine results from multiple
  studies in order in an effort to generalize
  findings.
• Meta-analysis of multiple studies of
  antidepressants let us estimate the relative
  importance of common factors (placebo effects)
  versus drug effects. 1-3

5
Drug effect accounted for 25 of measured
improvement
6
Evidenced based psychotherapy

• For several decades psychotherapy researchers
  have attempted to design randomly controlled
  trails (RCT) to investigate the effectiveness of
  specific methods of psychotherapy.
• Study design analogous to pharmacy trials, except
  that designing credible placebo treatments is
  much more problematic.
• Various treatment methods are being touted as
  evidenced based by citing the number of RCTs
  providing evidence that the treatment exceeded
  placebo (or some other treatment).

7
Psychotherapy brands

• The advocacy for the use of specific therapies is
  analogous to the advertising of brands of
  antidepressant medication.
• Calls for wide spread use of evidence based
  treatments in psychotherapy is analogous to the
  FDAs insistence that a drug may not be marketed
  for the treatment of depression until at least
  two studies have shown superiority to placebo.
• Advocates and practitioners of various evidence
  based treatments have a vested interest in
  discouraging the use of unproven treatments.

8
Brand differentiation

• Advocates of psychotherapy brands insist on the
  uniqueness of their therapy and the need to
  adhere to specific treatment procedures
• Research methodology requires the use of manuals
  and other techniques to standardize treatments
• Treatment effectiveness presumed to be dependent
  on the correct application of the active
  ingredients in the psychotherapy method.

9
The Dodo Bird Effect

• Rosenzweig S. (1936)
• Some implicit common factors in diverse methods
  of psychotherapy At last the Dodo said,
  Everybody has won and all must have prizes.
• Am J Orthopsychiatry 6412-5.

10
The Dodo Bird Lives!
Wampold BE, Mondin GW, Moody M, et al. (1997). A
meta-analysis of outcome studies comparing bona
fide psychotherapies Empirically, All must have
prizes. Psychol Bull 122203-15. Luborsky,
L., Rosenthal, R., Diguer, L., et al. 2002 The
dodo bird verdict is alive and well--mostly. J.
Psychotherapy Integration Vol 12(1) 32-57
11
Meta-analysis common factors

• Over two decades of meta-analytic studies have
  served to reinforce Rosenzweigs 1936 observation
  that different methods of psychotherapy tend to
  produce comparable outcomes the Dodo Bird
  Effect
• Lack of evidence for specific treatment effects
  bolster the argument that almost all of the
  effects of psychotherapy are due to factors
  common to all psychotherapies. 5-11

12
Real world example

• Human Affairs International (HAI) collected
  outcome data from a large number of clinicians
  between 1996 and 1998.
• Clinicians were asked to specify the primary
  method of psychotherapy (or medication management
  only)
• Analyses revealed no significant differences in
  the outcome or mean number of sessions across all
  treatment methods, including medication
  management.

13
Treatment outcomeHAI data
14
Fidelity and practicality

• Propagation of evidenced based treatment methods
  requires some method of measuring fidelity to the
  treatment.
• BIG PROBLEM!
• If clinicians reports using Cognitive Behavioral
  Therapy, how can we have any confidence that what
  that clinician did with a specific patient was
  comparable to the Cognitive Behavioral Therapy
  in the RCTs?

15
Goldilocks Effect

• Clinicians tend to be eclectic and flexible in
  their choice of treatment methods.
• Clinicians and patients tend to try different
  treatments until they find something that is
  just about right for them.
• Patients and clinicians tend to adjust the number
  and frequency of sessions depending on the
  patients level of distress and rate of
  improvement.
• Result All treatments appear to have similar
  outcomes and patients with good outcomes tend to
  use fewer session than patients with poor
  outcomes.

16
Recommended reading
Rigorous review and analyses of controlled
studies on psychotherapy outcome. Conclusion
much more variance resides with the clinician
than with the treatments.
17
Therapists effects

• Wampold and others argue that researchers have
  ignored the individual therapist as a source of
  variance.11, 16-24
• The person of the therapist is necessary to
  delivery the treatment, and personal
  characteristics of the therapist modify the
  effect of the treatment.
• Factors contributing to therapists effects may
  include elements clinical skill and knowledge as
  well as personality traits.

18
RCT and ANOVA brief history

• Some of the earliest applications of randomized
  control group design and analysis of variance
  were in agriculture and education. 12,13
• RCT methodology later adopted by medicine and
  eventually psychotherapy research. 11,14
• Simple ANOVA is appropriate only if the
  individual farmer, teacher or clinician has
  little or no impact on the effectiveness of the
  farming, teaching or treatment method!

19
HLM therapist effects

• Hierarchical Linear Modeling (HLM) is an advance
  in statistical methodology that permits us to
  model variance at the clinicians level and as
  well as the treatment level.
• An rapidly growing body of published research
  points to the conclusion that therapist effects
  almost certainly exceed specific treatment
  effects by a large margin.

20
Variance due to the clinician

• Published research making use of HLM points to
  the conclusion that the clinician accounts for
  much more of the variance in psychotherapy
  outcomes that treatment method per se. 11, 17-21
• Analyses of PacifiCare Behavioral Healths
  massive database database on patient outcomes
  confirms significant variance in psychotherapy
  outcomes at the clinician level. 24,25

21
PacifiCare Behavioral Health ALERT System

• Initiated an outcomes management program in 1998
  using 30 item patient self report questionnaires
  administered at regular intervals in treatment.
• ALERT System used to capture data and monitor
  patient outcomes in real time.
• Currently over 7,000 clinicians are contributing
  outcome data on a regular basis.
• Probably largest database on mental health
  outcomes in the world.

22
PBH research collaboration

• PBH actively sought the involvement of leading
  psychotherapy outcomes researchers from leading
  academic institutions.
• External researchers actively involved in design
  of the measurement system and ongoing analysis of
  the data.
• PBH encouraged publication of findings in
  academic journals.

23
The (almost) Bell Curve PBH data
Solo clinicians with sample sizes gt 20
24
Where is the variance?
25
of variance due to therapists in the real
world

• Analysis of PacifiCare Behavioral Health (PBH)
  data reveals 6 of variance due to therapist. 25
• Patients on medication have a higher of
  variance due the therapist than those receiving
  psychotherapy alone.
• Huh??

26
Therapists and meds
Outcomes (residualized scores) of 15 therapists
for patients with concurrent medication or no
medication 25
27
Test scores and medicationPBH data
28
Which treatment is best? Goldilocks Effect
Clients tend to get the treatment that is just
about right for them.
Normal functioning
Severe symptoms
29
Cross validation analysis

• Psychotherapists in PBH network ranked based on
  all cases from 1999-2002 if sample size gt30
  N116.
• If a therapists mean residualized final score lt
  0 then clinician rated Highly effective else
  clinician rated Less effective.
• Outcomes evaluated in the 2003-2004 cross
  validation period for a new sample of cases.

30
Cross validation results
31
Risk of not using HLM

• Wampold and colleagues at the University of
  Wisconsin recently reanalyzed data from the
  National Institute of Mental Healths Treatment
  of Depression Collaborative Research Program
  (TDCRP) study using HLM. 26-28
• Prior published reports found significant
  differences between two methods of psychotherapy
  as well as between placebo and antidepressant
  medication.
• Reanalysis of psychotherapy data using HLM
  revealed that 0 of the variance was due to the
  psychotherapy methods, while 8 was attributable
  to the therapists. 27

32
Psychiatrist effects

• Wampold and colleagues also used HLM to reanalyze
  the results antidepressant and placebo legs of
  the TDCRP study. 28
• Included the 9 individual psychiatrists as a
  variable.
• Outcome measured by change on patient self report
  measure (Beck Depression Inventory).
• 9.1 of the variance due to the psychiatrist
  only 3.4 due to the medication.
• Top third of psychiatrists achieved a better
  outcome with placebo than bottom third achieved
  with the antidepressant.

33
Placebo therapist effects

• Hypothesis Placebo/common factor effects are
  mediated by the clinician/patient relationship.
• Common factors tend to account for much more of
  the variance than specific treatment effects.
• If the effects of common factors are mediated by
  the clinician/patient relationship, then we would
  naturally find much of the variance in outcomes
  would be due to the clinician.
• The human factor matters!
• DUH!

34
Whats a clinician to do?

• If a wide variety of treatments appear to be
  equally efficacious, can a therapist do to
  achieve the best outcomes possible for their
  patients?
• A growing body of research supports the use of
  repeated administrations of patient self report
  outcome questionnaires to monitor response to
  treatment.
• Routine measurement and and early identification
  of patients with a poor response to treatment has
  been shown to reduce treatment failures. 29-36

35
Therapeutic alliance

• A large body of evidence suggests that the
  relationship and working alliance between the
  clinicians and patient is an important factor in
  the outcome. 39-45
• Routine use of a session rating/therapeutic
  alliance scale may permit clinicians to identify
  and repair problems in the working alliance.

36
Outcomes informed care

• Meta-method designed to improve outcomes across
  all patients and diagnoses, regardless of
  treatment method.
• Routine use of patient self report questionnaires
  to track symptom severity and therapeutic
  alliance.
• Use of feedback mechanisms to alert clinicians to
  patients at risk for poor outcomes.
• Performance feedback to clinicians, including
  comparison to outcomes to those of clinicians
  treating similar patients.
• Preferential referrals to highly effective
  clinicians.

37
2 case studies

• Resources for Living (RFL) provides telephonic
  EAP services. Customers include Walmart.
• Accountable Behavioral Healthcare Alliance (ABHA)
  is a managed behavioral healthcare organization
  servicing Oregon Health Plan members in 5 rural
  county area

38
Case history 1 RFL

• Began using the 4 item Outcome Rating and Session
  Rating Scales in 2002
• Administered over the phone as part of telephonic
  counseling sessions.
• Baseline data collected for 5 months
• Baseline data used to create trajectory of change
  graphs
• Real time feedback provided to counselors via
  SIGNAL System

39
RFL Signal System results
Training and feedback
Baseline period
40
Case history 2 ABHA

• Began utilizing the 4 item Oregon Change Index
  (OCI) in 2004.
• OCI administered at every session in outpatient
  and day treatment settings.
• OCIs collected at over 80 of all sessions.
• Collected baseline data for 18 months, began
  giving feedback in mid 2005.
• Updated Excel based Active Case Report contains
  outcome data on all cases seen within the last 6
  weeks is emailed to the clinicians weekly.

41
ABHA results
42
Implications for clinicians

• Good news The clinician matters!!!!!!
• All treatments (including medications!?) are only
  as effective as the clinicians delivering the
  treatment.
• Clinicians have an ethical responsibility to
  assess and improve their personal effectiveness
  as clinicians they cannot rely on the treatments
  alone to be curative.
• Effective clinicians deliver high value services
  and are worth more money!!!

43
Implications for administrators policy makers

• Exclusive focus on the effectiveness of
  treatments rather than the value of the
  clinicians limits the potential to improve
  outcomes.
• Use of effective clinicians tends to lower costs.
• Administrators and policy makers have an
  obligation to consumers to assure that they have
  access to effective clinicians.
• Failure to monitor outcomes at the clinician
  level places consumers at risk.

44
References

• Kirsch, I Sapirstein, G. 1998. Listening to
  Prozac but hearing placebo A meta analysis of
  antidepressant medication. Prevention
  Treatment. 1, Article 0002a, No Pagination
  Specified
• Kirsch, I. 2000. Are drug and placebo effects in
  depression additive? Biological Psychiatry 47,
  733-73.
• Kirsch, I, Moore, TJ, Scoboria, A, Nicholls, SS.
  2002. The emperor's new drugs An analysis of
  antidepressant medication data submitted to the
  U.S. Food and Drug Administration. Prevention
  Treatment. 5(1), No Pagination Specified
• Rosenzweig S. 1936. Some implicit common factors
  in diverse methods of psychotherapy At last the
  Dodo said, Everybody has won and all must have
  prizes. Am J Orthopsychiatry 6412-5.
• Shapiro DA Shapiro D. 1982. Meta-analysis of
  comparative therapy outcome studies A
  replication and refinement. Psychol Bull
  92581-604.

45
References (continued)

• Robinson LA, Berman JS, Neimeyer RA. 1990.
  Psychotherapy for treatment of depression A
  comprehensive review of controlled outcome
  research. Psychol Bull 10830-49.
• Wampold BE, Mondin GW, Moody M, et al. 1997. A
  meta-analysis of outcome studies comparing bona
  fide psychotherapies Empirically, All must have
  prizes. Psychol Bull 122203-15.
• Ahn H, Wampold BE. 2001. Where oh where are the
  specific ingredients? A meta-analysis of
  component studies in counseling and
  psychotherapy. J Counsel Psychol 48251-7.
• Chambless DL, Ollendick TH. 2001. Empirically
  supported psychological interventions
  Controversies and evidence. Annual Rev Psychol
  52685-716.
• Luborsky, L., Rosenthal, R., Diguer, L., et al.
  2002. The dodo bird verdict is alive and
  well--mostly. J. Psychotherapy Integration Vol
  12(1) 32-57

46
References (continued)

• Wampold BE. 2001. The great psychotherapy debate
  Models, Methods, and Findings. Mahwah NJ
  Lawrence Erlbaum Associates. Wampold BE, Mondin
  GW, Moody M, et al. 1997. A meta-analysis of
  outcome studies comparing bona fide
  psychotherapies Empirically, All must have
  prizes. Psychol Bull 122203-15.
• McCall, WA. 1923 How to experiment in education.
  New York McmIllan.
• Fisher, RA. 1935 The design of experiments.
  Edinburgh Oliver and Boyd.
• Gehan, E. Lemark, NA. 1994. Statistics in
  medical research Developments in clinical
  trials. New York Plenum Press.
• Martindale C. 1978. The therapist-as-fixed-effect
  fallacy in psychotherapy research. J Consult Clin
  Psychol 461526-30.

47
References (continued)

• Luborsky L, Crits-Christoph P, McLellan T, et al.
  1986. Do therapists vary much in their success?
  Findings from four outcome studies. Am J
  Orthopsychiatry 56501-12.
• Crits-Christoph P, Baranackie K, Kurcias JS, et
  al. 1991. Meta-analysis of therapist effects in
  psychotherapy outcome studies. Psychother Res
  181-91.
• Crits-Christoph P, Mintz J. 1991. Implications of
  therapist effects for the design and analysis of
  comparative studies of psychotherapies. J Consul
  Clin Psychol 5920-6.
• Wampold BE. 1997. Methodological problems in
  identifying efficacious psychotherapies.
  Psychother Res 721-43,
• Elkin I. 1999. A major dilemma in psychotherapy
  outcome research Disentangling therapists from
  therapies. Clin Psychol Sci Prac 610- 32.

48
References (continued)

• Wampold BE, Serlin RC. 2000. The consequences of
  ignoring a nested factor on measures of effect
  size in analysis of variance designs. Psychol
  Methods 4425-33.
• Huppert JD, Bufka LF, Barlow DH, et al. 2001.
  Therapists, therapist variables, and
  cognitive-behavioral therapy outcomes in a
  multicenter trial for panic disorder. J Consul
  Clin Psychol 69747-55.
• Okiishi J, Lambert MJ, Nielsen SL, et al. 2003.
  Waiting for supershrink An empirical analysis of
  therapist effects. Clin Psychol Psychother
  10361-73.
• Brown GS, Jones ER, Lambert MJ, et al. 2005.
  Identifying highly effective psychotherapists in
  a managed care environment. Am J Managed Care
  11(8)513-20.
• Wampold BE, Brown GS. 2005. Estimating
  variability in outcomes due to the therapist A
  naturalistic study of outcomes in managed care. J
  Consul Clin Psychol. 73(5) 914-923.

49
References (continued)

• Elkin, I, Shae, T, Watkins, JT., et al. 1989.
  National Institute of Mental Health Treatment of
  Depression Collaborative Research Program
  General effectiveness of treatments. Archive of
  General Psychiatry. 46 971-982.
• Kim DM, Wampold BE, Bolt DM. 2006. Therapist
  effects and treatment effects in psychotherapy
  Analysis of the National Institute of Mental
  Health Treatment of Depression Collaborative
  Research Program. Psychother Res. 16(2) 161-172.
• McKay, KM, Imel, ZE Wampold, BE. In press.
  Psychiatrist effects in the pharmacological
  treatment of depression. J. Affective Disorders.
• Hannan C, Lambert MJ, Harmon C et al. 2005. A lab
  test and algorithms for identifying clients at
  risk for treatment failure. J Clin Psychol
  61(2)155-63.
• Lambert MJ, Harmon C, Slade K et al. 2005.
  Providing feedback to psychotherapists on their
  patients progress Clinical results and practice
  suggestions J Clin Psychol 61(2)165-74.

50
References (continued)

• Harmon C, Hawkins, Lambert MJ et al. 2005.
  Improving outcomes for poorly responding clients
  The use of clinical support tools and feedback to
  clients. J Clin Psychol 61(2)175-85.
• Brown GS, Jones DR. 2005. Implementation of a
  feedback system in a managed care environment
  What are patients teaching us? J Clin Psychol
  61(2)187-98.
• Claiborn CD, Goodyear EK. 2005. Feedback in
  psychotherapy. J Clin Psychol 61(2)209-21.
• Lueger RJ. 1998. Using feedback on patient
  progress to predict the outcome of psychotherapy.
  J Clin Psychol 54383-93.
• Lambert MJ, Whipple JL, Smart DW, et al. 2001.
  The effects of providing therapists with feedback
  on patient progress during psychotherapy Are
  outcomes enhanced? Psychother Res 11(1)49-68.

51
References (continued)

• Lambert MJ, Whipple JL, Vermeersch DA, et al.
  2002. Enhancing psychotherapy outcomes via
  providing feedback on client progress A
  replication. Clin Psychol Psychother 991-103.
• Whipple JL, Lambert MJ, Vermeersch DA, et al.
  2003. Improving the effects of psychotherapy The
  use of early identification of treatment failure
  and problem-solving strategies in routine
  practice. J Counsel Psychol 50(1)59-68.
• Lambert MJ, Whipple JL, Hawkins EJ, et al. 2003.
  Is it time for clinicians to routinely track
  patient outcome? A meta-analysis. Clin Psychol
  Sci Prac 10288-301.
• Bachelor, A., Horvath, A. (1999). The
  therapeutic relationship. In M.A. Hubble, B.L.
  Duncan, and S.D. Miller (eds.). The Heart and
  Soul of Change What Works in Therapy.
  Washington, D.C. APA Press, 133-178.
• Blatt, S. J., Zuroff, D.C., Quinlan, D.M.,
  Pilkonis, P. (1996). Interpersonal factors in
  brief treatment of depression Further analyses
  of the NIMH Treatment of Depression Collaborative
  Research Program. J Consul Clin Psychol. 64,
  162-171.

52
References (continued)

• Bordin, E. S. (1979). The generalizability of the
  psychoanalytic concept of the working alliance.
  Psychotherapy Theory, Research, and Practice,
  16, 252-260.
• Burns, D., Nolen-Hoeksema, S.  (1992).
  Therapeutic empathy and recovery from depression
  in cognitive-behavioral therapy A structural
  equation model. J Consul Clin Psychol. 60,
  441-449.
• Connors, GJ, DiClemente, CC., Carroll, KM, et al.
  1997 The therapeutic alliance and its
  relationship to alcoholism treatment
  participation and outcome. J Consul Clin Psychol,
  65(4), 588-598.
• Horvath, A. O., Symonds, B. D. (1991). Relation
  between working alliance and outcome in
  psychotherapy A meta-analysis. J Consul Clin
  Psychol. 38, 139-149.
• Krupnick, J., Sotsky, SM, Simmens, S et al. 1996.
  The role of the therapeutic alliance in
  psychotherapy and pharmacotherapy outcome
  Findings in the National Institute of Mental
  Health Treatment of Depression Collaborative
  Research Project. J Consul Clin Psychol. , 64,
  532-539.

53
About the presenter

• G.S. (Jeb) Brown is a licensed psychologist with
  a Ph.D. from Duke University. He served as the
  Executive Director of the Center for Family
  Development from 1982 to 19987. He then joined
  United Behavioral Systems (an United Health Care
  subsidiary) as the Executive Director for of
  Utah, a position he held for almost six years.
  In 1993 he accepted a position as the Corporate
  Clinical Director for Human Affairs International
  (HAI), at that time one of the largest managed
  behavioral healthcare companies in the country.
• In 1998 he left HAI to found the Center for
  Clinical Informatics, a consulting firm
  specializing in helping large organizations
  implement outcomes management systems. Client
  organizations include PacifiCare Behavioral
  Health/ United Behavioral Health, Department of
  Mental Health for the District of Columbia,
  Accountable Behavioral Health Care Alliance,
  Resources for Living and assorted treatment
  programs and centers throughout the world.
• Dr. Brown continues to work as a part time
  psychotherapist at behavioral health clinic in
  Salt Lake City, Utah. He does measure his
  outcomes.

54
http//www.clinical-informatics.com jebbrown_at_clin
ical-informatics.com 1821 Meadowmoor Rd. Salt
Lake City, UT 84117 Voice 801-541-9720