An Evidence-Based Approach to Clinical Practice in Communication Disorders

1
An Evidence-Based Approach to Clinical Practice
in Communication Disorders

• Chris Dollaghan
• University of Pittsburgh
• dollagha_at_csd.pitt.edu
• 2004 Donalda J. McGeachy Memorial Lecture
  University of Toronto
• May 28, 2004

2
Evidence-Based Practice

• the conscientious, explicit, and judicious
  use of current best evidence in making decisions
  about the care of individual patients . . . by
  integrating individual clinical expertise with
  the best available external clinical evidence
  from systematic research.
• (Sackett, Rosenberg, Gray, Haynes, Richardson,
  1996 71)

3
N. B

• Evidence is never enough (Guyatt et al., 2000)
• Not all evidence is relevant to clinical
  questions Basic research into underlying
  mechanisms of disease is a necessary but
  insufficient guide to clinical practice (EBM
  Working Group, 1992)

4
Why EBP?

• The first diagnostic manual (Kraemer Sprenger,
  1487 cited in Meehl, 1997)

5
How could well-meaning experts, using best
technical evidence of their age, be so wrong?

• Clinical decisions were based on belief, opinion,
  and past experience that had not been subjected
  to systematic, unbiased, and reproducible (i.e.,
  scientific) test.
• Francis Bacons observation
  (1620 cited in Meehl, 1997)
• The human understanding, once it has adopted
  opinions, either because they were already
  accepted and believed, or because it likes them,
  draws everything else to support and agree with
  them.

6
Starting point for EBP

• Acknowledgement that humans inherently prefer
  what they already believe, and thus that strong
  evidence cannot come without strong tests of such
  beliefs and preferences, regardless of the
  eminence of those holding them

7

• Clinical experience and expert opinion can be
  important starting points, but must be tested
  systematically, in an unbiased fashion, rather
  than being accepted at face value
• Even (especially!) when they stand to reason

8
Seven steps to EBP (adapted from Sackett et al.,
2000)

1. Formulate a foreground question
2. Find best current evidence
3. Critical appraisal
4. Evidence summary via CAT
5. Decide whether the evidence is strong enough to
   influence your clinical practice
6. Integrate the evidence with the intangibles
7. Update!

9
Step 1 Formulate a foreground question

• The foreground-background distinction
• Background questions
• ask for general knowledge about a condition, via
  a question word (who, what, where, when, how,
  why)
• are more frequent when our experience with a
  disorder is limited
• relative proportion of background questions
  decreases as clinical experience increases

10
Foreground questions

• The focus for EBP
• Ask a quite specific, answerable question about
  the best way to diagnose, prognose, or treat
• Have four essential components (PICO)

11
Formulating a foreground question PICO (SIGN
group, 2000)

• P (the patient and/or problem of interest)
• I (the intervention, defined broadly to
  encompass clinical decisions about diagnosing,
  treating, prognosticating, etc.)
• C (the comparison intervention)
• O (the clinical outcome of interest)

12

• Clinical question about treatment
• In toddlers with delayed language, does
  parent-administered treatment result in
  significantly greater language gains than no
  treatment?
• Clinical question about diagnosis
• Does Test A identify four-year-olds with
  language disorders significantly more accurately
  than a language sample analysis?

13
Step 2 Find ostensibly best current evidence
(i.e., candidate evidence)

• Cost-benefit ratio for various potential sources
  of evidence (e.g., colleagues, consensus
  statements by respected authorities, notes from
  classes or workshops, textbooks, professional
  journals, previous experience with similar cases,
  current studies in high-yield journals)

14
Some ways of finding high quality evidence, or
letting it find you

• PubMed demonstration www.pubmed.com (Please do
  the tutorial!)
• Clinical query function
• Dx
• Tx
• Meta-analysis or systematic review
• Cubby
• www.guideline.gov and other regular updates
• Guiltless reliance on other sources if strong
  evidence does not exist

15
Finding ostensibly best evidence means knowing
how to judge evidence. . .
16
Step 3 Critical appraisal (Sackett et al. 2000)

• Applying explicit criteria to judge evidence
  quality
• Criteria vary slightly according to whether
  evidence comes from a study of diagnosis,
  therapy, prognosis, meta-analysis, etc.
• Three consistent themes
• Validity (avoid bias and confounding)
• Importance (effect sizes and impact)
• Precision (confidence intervals)

17
Evaluating validity An overview

• Avoiding confounding
• Design (prospective, controlled, random
  assignment to groups)
• Full disclosure of patient enrollment and loss
• Avoiding bias
• Blinding (masking, concealment)
• Unblinded studies of treatment have effects an
  average of 40 larger than blinded studies
  (Schulz Chalmers, 1995)

18
Design types and definitions (note that validity
depends on more than design)

• Experimental patients are enrolled
  prospectively, randomized to group/condition, and
  some variable is controlled (i.e., some
  manipulation is imposed by the investigator)
• Quasi-experimental prospective but not
  randomized or controlled
• Non-experimental patients are identified
  retrospectively, no manipulation of variables

19
Randomized, controlled trial (RCT)

• Study is designed before patients are enrolled
• Patients are assigned at random to one of the
  groups to be compared.
• Pros If randomization is accomplished correctly,
  best odds that the groups being compared do not
  differ systematically on some variable other than
  the one of interest (treatment vs. no-treatment)
• Cons Costly, time-consuming, in some cases may
  raise ethical issues potential for volunteer bias

20
Quasi-experimental prospective but without
random assignment

• E.g., Cohort study Patients with and without a
  variable of interest are identified and followed
  forward in time to compare their outcomes
• Pros Ability to study low-incidence and/or
  late-emerging problems efficiently
• Cons Exposure may be linked to a hidden
  confounder blinding is difficult

21
Non-experimental studies

• Cross-sectional/correlational Group is observed
  at a single point in time exposure and outcome
  are determined simultaneously
• Pros Cheap, ethically safe
• Cons Causality cant be established,
  susceptible to recall bias, confounders may not
  be equally distributed
• Case-control Patients with and without an
  outcome are identified post hoc, and previous
  exposure to a variable of interest is compared
• Pros May be the only feasible method for very
  rare disorders, or if theres a long lag between
  exposure and outcome
• Cons Recall bias, reliance on retrospective
  records to determine exposure, potential
  confounding

22
Weakest non-experimental designs

• Case series No control group
• Pros Readily accessible to practitioners
• Cons Nothing improves results like no control
  group (Barrett-Connor, 2002) due to difficulty of
  controlling for bias and confounders
  generalizability unknown.
• Case report N 1
• Pros Readily accessible
• Cons See above, and limited external validity
  (generalizability)

23

• Weaker study designs can be critically important
  in the early stages of investigation of a
  clinical question (e.g., Robey, 2003)
• We just need to recognize their limitations for
  making strong inferences about clinical decisions

24
Evaluating validity II Avoiding bias

• Bias is virtually inescapable even when
  inadvertent
• Cues resulting in unblinding can be very subtle
• Can affect randomization
• Can affect evaluation
• Treating clinician cant be the evaluator of the
  patient
• Blinded evaluators should rate variety of cases
  and controls, without knowing which is which
• Blinding might require ratings of , e.g.,
  de-identified recordings randomized to ensure
  that stage of treatment cannot be inferred from
  them

25
Example 1 Adequate blinding?

• Tx target increased social participation as
  indexed by number of utterances
• Participants were assessed initially by the
  clinician, and then randomly assigned to receive
  either no treatment or 3 treatment sessions/week
• After 12 weeks, all were re-assessed and the tx
  group was found to talk significantly more than
  control group

26

• No Subtle (or not-so-subtle) expectations of
  childs performance based on experiences in
  treatment with the clinician could affect his or
  her evaluation of outcome.

27
Example 2 Adequate blinding?

• Study of two therapies to improve vocal quality
• Patients were identified prospectively and
  randomly assigned to receive Tx 1 or Tx 2
• Observers blinded to group assignment rated vocal
  quality prior to treatment phase, after final
  treatment session, and for a sample obtained 3
  months after treatment phase

28

• No Time of measurement (pre-tx, immediate
  post-tx, and long post-tx) was not concealed from
  raters, so their expectations of tx impact could
  have inadvertently influenced their ratings

29
Example 3 Adequate blinding?

• Clinicians in early intervention settings were
  invited to refer patients with a diagnosis of
  childhood apraxia of speech to the study
• All referred children were assessed with the new
  test, by an examiner who was unaware of the
  childs referring site.

30

• Not if evaluator knew that all had been diagnosed
  previously expectations associated with
  evaluating a child with known apraxia could bias
  evaluation (e.g., cause increased focus on
  expected symptoms and decrease attention to other
  characteristics).

31
End of overview of first theme evaluating
validity On to second theme (evaluating
importance)