EBC course

1
Critical Appraisal of the Clinical Literature
The Big Picture

• Cynthia R. Long, PhD
• Associate Professor
• Palmer Center for Chiropractic Research

2
Outline

• Overall purpose of critical appraisal
• Consider levels of evidence
• Some tips on interpreting significant findings
• Potentially useful references

3
Overall Purpose

• To critically appraise a research article in
  order to assess the validity of the authors
  conclusions

4
Sections of an Article

• Introduction Background and explanation of
  rationale for study.
• Methods How was study done? Should allow study
  to be replicated.
• Results Report results (data).
• Discussion Interpret results. Draw conclusions
  from results.
• Abstract Article summary.

5
Recommendation

• Read the abstract last when familiarizing
  yourself with critically appraising the
  literature.
• Assess evidence from reading the article, not the
  abstract.
• Read the abstract in deciding whether or not you
  are interested in the topic of the article.

6
Critical Appraisal

• similar across types of studies for Introduction
  and Discussion sections
• information in Methods sections may differ
• information in Results sections may differ

7
Overall Questions to Ask

• Is the study design appropriate to address the
  research question?
• In the Discussion Section Are the findings...
• ...consistent with the research question of the
  study?

8
Terminology Types of Clinical Studies

• Risk
• Diagnostic
• Prognostic
• Intervention

9
TerminologyClinical Intervention Studies

• Evaluates which treatment interventions are most
  useful and effective for a given clinical
  condition

10
Clinical Intervention Studies

• One-group Pre/post Study
• Pilot RCT
• Randomized Clinical Trial (RCT)
• Systematic Review of RCTs

11
Levels of Evidence
12
Levels of Evidence

• Is the study design appropriate to address the
  research question?
• In the Discussion Section Are the findings...
• ...consistent with the research question of the
  study?

13
One-group pre/post design

• Addresses do patients improve after treatment?
• Cant address is treatment effective for
  patients?
• No comparison group
• Patients may improve natural course of
  condition, in study, change in lifestyle,
  treatment

14
One-group pre/post design Example

• Hawk C, Long CR, Azad A. Chiropractic Care for
  Women with Chronic Pelvic Pain A Prospective
  Single-Group Intervention Study. JMPT
  19972073-79.
• Results women improved!
• Conclusion needs further study

15
Pilot RCT

• Addresses is the RCT feasible?
• Cant address is treatment effective for
  patients?
• Not powered (i.e. sample size not large enough)
• Note often only published if RCT is determined
  to be unfeasible

16
Pilot RCT Example 1

• Hawk C, Long CR, et al. Issues in planning a
  placebo-controlled trial of manual methods
  Results of a pilot study. J of Alt Comp Med
  2002821-32.
• Results recruitment not feasible
  standardization of treatment protocol difficult
  among multiple sites
• Conclusion put on hold

17
Pilot RCT Example 2

• Bronfort G, et al. Nonoperative treatments for
  sciatica A pilot study for a randomized clinical
  trial. JMPT 200023536-544.
• Results recruitment not feasible!
• Conclusion put on hold

18
RCT

• Addresses is the treatment effective for
  patients?
• Caveats
• Must be a powered study
• sample size must be formally justified in the
  Methods section (based on effect sizeminimally
  important clinical difference variability of
  outcome measure statistical test)

19
RCT Example

• Hurwitz E, et al. A randomized trial of
  chiropractic manipulation and mobilization for
  patients with neck pain Clinical outcomes from
  the UCLA neck-pain study. Am J Public Health
  2002921634-1641.
• Results no statistically or clinically
  significant differences among groups
• Conclusion cervical spine mobilization is as
  effective as manipulation in reducing neck pain
  and related disability among chiropractic
  patients

20
Overall Questions to Ask

• In the Discussion Section Are the findings...
• ...consistent with the research question of the
  study?
• consistent with the results presented?
• given in the context of current evidence?

21
Systematic Review of RCTs

• Addresses is the treatment effective for
  patients?
• Looks at all RCTs of the treatment for patients
  and combines based on quality of original RCTs

22
RCT Example

• Bronfort G, et al. Efficacy of Spinal
  Manipulation for Chronic Headache A Systematic
  Review. JMPT 200124457-466.
• Conclusions cant make firm conclusions (few
  trials of adequate methodological quality)

23
RCT Example

• Conclusions
• Cervicogenic SMT better effect than massage
• Tension-type and migraine SMT effect comparable
  with commonly used first-line prophylactic
  prescription meds

24
Terminology Descriptive Statistics (Results)

• Patient characteristics at baseline
• Examples mean, standard deviation, median,
  range, percentage
• Assess group comparability on baseline
  characteristics
• Assess generalizability of results to target
  population

25
Terminology Analytical Statistics (Results)

• Assess statistical significance with confidence
  intervals and p-values
• Within and between group differences
• Make inference about target population
• Must be appropriately interpreted in the context
  of the research question and the study design

26
Terminology

• P-values and confidence intervals
• Reflects measure of effect relative to variation
  and sample size

27
Statistical Significance

• Interpreting p-values
• plt0.01 ? statistically significant difference!
• 0.01?p?0.05 ? statistically significant
  difference
• 0.05ltp?0.10 ? borderline statistically
  significant difference
• pgt0.10 ? no statistically significant difference

28
Significant Findings

• jargon term
• Need to consider BOTH statistical and clinical
  significance

29
Clinical Significance

• i.e. clinical importance
• is defined before study is conducted
• assessed with descriptive statistics (e.g. mean
  improvement in outcome measure)

30
Possible Scenarios

• statistically and clinically significant findings
• clinically significant, but not statistically
  significant
• statistically significant, but not clinically
  significant

31
Critical Appraisal References

• Useful? It depends
• BMJ series available for free on bmj.com
• JAMA articles?
• Chiropractic Research Review