The golden standard or fools gold

1

• The golden standard or fools gold
• Selective data, impact on safety

Clinical trials on trial HAI Europe Open Seminar
2008 Neues Stadthaus, Berlin, 21 November 2008
2
RCTs The golden standard or fools gold?

• Internal validity
• External validity - Efficacy vs Effectiveness
• Publication bias
• Fraud

3
Internal validity

• Did the control group receive optimal treatment?

4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Internal validity

• Did the control group receive optimal treatment?

• Was the dose of the control group adequate?

9
12 mg haloperidol
gt 12 mg haloperidol
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
Favours atypical
Favours haloperidol
Drop out rates by dose of comparator drug in
trials of patients with schizophrenia or related
disorders (risk difference and 95 confidence
intervals)
Geddes et al., 2000
10
Internal validity

• Did the control group receive optimal treatment?

• Was the dose of the control group adequate?

• Was the sample size adequate to identify any
  relevant difference?

11
Internal validity

• Did the control group receive optimal treatment?

• Was the dose of the control group adequate?

• Was the sample size adequate to identify any
  relevant difference?

• Did the published results refer to the primary
  variable?

12
(No Transcript)
13
(No Transcript)
14
Internal validity

• Did the control group receive optimal treatment?

• Was the dose of the control group adequate?

• Was the sample size adequate to identify any
  relevant difference?

• Did the published results refer to the primary
  variable?

• Have all the trial results been published?

15
(No Transcript)
16
(No Transcript)
17
Internal validity

• Did the control group receive optimal treatment?

• Was the dose of the control group adequate?

• Was the sample size adequate to identify any
  relevant difference?

• Did the published results refer to the primary
  variable?

• Have all the trial results been published?

• Were the results presented as a relative risk
  reduction, or as an absolute risk reduction?

18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
RCTs The golden standard or fools gold?

• Internal validity
• External validity - efficacy vs effectiveness
• Publication bias
• Fraud

23
External validity of clinical trials

• Context
• Reference population
• Selection criteria
• Diagnostic criteria
• Follow up
• Treatments (doses, compliance)
• Duration
• Primary and other variables
• Adverse effects

Lancet 2005 365 82-93
24
(Un)transferability to clinical practice

• Patients in RCTs differ from those in real
  practice
• Age
• Comorbidity
• Other treatments
• Doses taken, compliance
• Diagnostic criteria

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Efficacy vs effectiveness

• RCT UCP
• Nº of patients 102-103 104-107
• Duration Short Longer
• Populations High risk groups
  Potentially the
• excluded whole
  population
• Comorbidity Generally excluded Often present
• Conditions Well defined Ill-defined
• Nº of drugs One or limited
  Undetermined
• Dose/dosage Generally constant Often
  variable
• Pattern of use Continuous Intermittent
• Follow up Careful Less careful

31
RCTs The golden standard or fools gold?

• Internal validity
• External validity - efficacy vs effectiveness
• Publication bias
• Fraud

32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
RCTs The golden standard or fools gold?

• Internal validity
• External validity - efficacy vs effectiveness
• Publication bias
• Fraud

36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
Third Report of the Expert Panel on Detection,
Evaluation, and Treatment of High Blood
Cholesterol in Adults (Adult Treatment Panel III)
(ATP III)
Financial Disclosure Dr Grundy has received
honoraria from Merck, Pfizer, Sankyo, Bayer, and
Bristol-Myers Squibb. Dr Hunninghake has current
grants from Merck, Pfizer, Kos Pharmaceuticals,
Schering Plough, Wyeth Ayerst, Sankyo, Bayer,
AstraZeneca, Bristol-Myers Squibb, and G. D.
Searle he has also received consulting honoraria
from Merck, Pfizer, Kos Pharmaceuticals, Sankyo,
AstraZeneca, and Bayer. Dr McBride has received
grants and/or research support from Pfizer,
Merck, Parke-Davis, and AstraZeneca has served
as a consultant for Kos Pharmaceuticals, Abbott,
and Merck and has received honoraria from
Abbott, Bristol-Myers Squibb, Novartis, Merck,
Kos Pharmaceuticals, Parke-Davis, Pfizer, and
DuPont. Dr Pasternak has served as a consultant
for and received honoraria from Merck, Pfizer,
and Kos Pharmaceuticals, and has received grants
from Merck and Pfizer. Dr Stone has served as a
consultant and/or received honoraria for lectures
from Abbott, Bayer, Bristol-Myers Squibb, Kos
Pharmaceuticals, Merck, Novartis,
Parke-Davis/Pfizer, and Sankyo. Dr Schwartz has
served as a consultant for and/or conducted
research funded by Bristol-Myers Squibb,
AstraZeneca, Merck, Johnson Johnson-Merck, and
Pfizer.
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
Conclusions

• The RCT is the best epidemiological method for
  causal inference
• However, it is often performed in a way which
  favours the sponsors treatment
• In its design
• In data analysis and interpretation
• In the publication of results
• At best, RCTs are one of many pieces of evidence
  about therapeutic interventions

47
Conclusions

• The appraisal of innovation should not only take
  into account the so-called EBM, but also other
  evidence
• Pharmacodynamics
• Pharmacokinetics
• Availability of therapeutic alternatives
• The medical literature is no longer reliable for
  valid information
• Research should be performed, analyzed and
  published in a way which should be independent
  from commercially interested parties