External Validity of Trials - PowerPoint PPT Presentation

About This Presentation
Title:

External Validity of Trials

Description:

An early RCT of chiropractic manipulation for back pain was criticised by physiotherapists that the chiropractic was undertaken in private practice. ... – PowerPoint PPT presentation

Number of Views:37
Avg rating:3.0/5.0
Slides: 20
Provided by: davidto4
Category:

less

Transcript and Presenter's Notes

Title: External Validity of Trials


1
External Validity of Trials
2
Background
  • External or ecological validity refers to whether
    the results of the trial can be generalised to
    the general clinical population.
  • Randomisation per se does NOT make a study
    externally valid.

3
Validity Issues
  • Are the correct patients being included in the
    trials?
  • Are the correct practitioners taking part in
    the study?
  • Are the correct facilities being used?

4
Trial Problems
  • Many if not most trials are undertaken in
    circumstances that are far removed from usual
    clinical practice.
  • Usually clinicians running trials are content
    experts yet the results are to be applied by
    non-experts.

5
Osteoporosis Trials
  • Most of the large osteoporosis trials with
    fracture as an outcome recruited participants
    from bone clinics.
  • Patients under care of clinical experts but
    results are expected to be applied by clinicians
    who are not experts.

6
Participants
  • Usually participants who go into trials are
    different from those who do not.
  • These storm troopers of patients usually comply
    with instructions better than average.
  • Combined with different clinicians and patients
    results can be different from the real world.

7
Trial participants v non
 
 
8
Crude risk of fracture
 
9
Risk of fracture (adjusted)
 
10
Alendronate
  • Alendronate is a bisphosphonate for osteoporosis
    treatment. Large trials conducted in expert
    clinical centres showed it reduced fractures with
    few or no side-effects.
  • Things were different when it was licensed and
    real clinicians used the drug.

11
Alendronate problems
  • To take a bisphosphonate one needs to take it on
    an empty stomach with a large glass of water and
    remain upright for at least 2 hours.
  • Otherwise the drug can get affect the oesophagus
    and cause ulceration.
  • In real life this is what happened and led to a
    warning. Trial data gave to evidence this was a
    problem.

12
The Wrong Solution
  • Often once the phase III trial has been finished
    companies finance a real world non-randomised
    study.
  • This is usually nothing more than a marketing
    exercise and produces worthless data on real
    world experience of the problem.

13
A correct solution
  • Involves some form of randomisation to eliminate
    confounding.
  • For example one might undertake a cluster trial
    of giving the drug non-specialist physicians to
    use of their patients BEFORE the drug is widely
    available.
  • This will produce more robust data.

14
Alternatively
  • We should try and design our trials to be as
    pragmatic as possible so that the results are
    widely generalisable.
  • This means recruiting non-expert clinicians and
    their patients.
  • Using a pragmatic design rather than an
    explanatory design.

15
Back Pain Trials
  • An early RCT of chiropractic manipulation for
    back pain was criticised by physiotherapists that
    the chiropractic was undertaken in private
    practice.
  • May have been a Bach effect.

16
MRC back pain trial
  • To address the issue of nice premises the MRC
    back pain trial included an arm where patients
    were randomised to be treated on private premises
    or NHS premises.
  • Increased the costs of the trial hugely.

17
Surgical Trials
  • Often surgical trials have poor external validity
    because the new technique is usually developed in
    a teaching hospital among expert surgeons (e.g.
    laparoscopic methods).
  • It is unlikely the results of surgery undertaken
    in top hospitals would be the same as a
    bog-standard DGH.

18
Big and Simple
  • Try and make the trial large, as it allows for
    various PRE-SPECIFIED sub-group analyses
    (specialists vs generalists).
  • Simple which means one can get as many in as
    possible.

19
Summary
  • Most trials will NEVER be able to recruit exactly
    the same types of participants as those who
    receive treatment in clinical practice.
  • PRAGMATIC trials have the strongest external
    validity and usually one of these should be done
    before implementation.
Write a Comment
User Comments (0)
About PowerShow.com