Publication bias in clinical trials

1
Publication bias in clinical trials

• Kamran Abbasi
• Deputy editor, BMJ

2
Merhaba
3
I want to talk about . . .

• What is publication bias?
• Why does it matter?
• What is the evidence for it?
• What can be done about it?
• How has the BMJ responded?

4
There are many types of bias

• Selection bias biased allocation to comparison
  groups
• Performance bias unequal provision of care
  except treatment being evaluated
• Detection bias biased assessment of outcome
• Attrition bias biased occurrence and handling of
  deviations from protocol and loss to follow up
• . . . and on and on
• (From Egger et al BMJ 200132342-46 (7 July)

5
What is publication bias (1)?

• A definition
• Publication bias refers to the greater
  likelihood that studies with positive results
  will be published
• JAMA
  20022872825-2828

6
What is publication bias (2)?

• An alternative definition
• Publication bias is the selective or multiple
  publication or suppression of trial results so
  that the scientific record is distorted

7
Why does it matter?

• Distorts the scientific record
• Hides the truth
• Influences doctors decision making
• Misleads policy makers
• Causes harm to patients
• Costly for the health service
• A form of scientific and research misconduct

8
Who is to blame?

• Wicked researchers?
• Very wicked sponsors?
• Editors the wickedest of all?
• (and lets not forget reviewers)

9
What is the evidence for it (1)?

• Stern and Simes BMJ 1997315640-645
• Question To what extent is publication
  influenced by study outcome?
• Studies submitted to an Australian ethics
  committee over 10 years
• Examined protocols
• Questionnaire to authors (70 response)

10
Stern and Simes results
Clinical trials (n130)
All studies (n520)
Positivegt negative
3.13 (1.76 to 5.58)
2.32 (1.47 to 3.66)
4.7 vs 8.0 yrs
4.8 vs 8.0 yrs
Time to publication
11
Stern and Simes conclusions

• Positive trials are more likely to be submitted
  for publication
• Positive trials are more likely to be published
• Positive trials are more likely to be published
  quickly
• Implications for systematic reviews
• Important to register all trials

12
What is the evidence for it (2)?

• Lexchin and Bero BMJ 20033261167-70
• Question Does drug industry sponsorship
  influence research quality and outcome?
• Meta-meta-analysis
• Industry research less likely to be published
  (more likely in symposium proceedings)
• No difference in methodological quality
• More likely to have a positive finding
  (OR 4.05 95 CI 2.98 to 5.51)

13
Lexchin and Bero

• A wide range of diseases
• eg osteoarthiritis of the knee, multiple
  myeloma, psychiatric problems, Alzheimers
  disease, venous thromboembolism
• A wide range of drugs
• eg tacrine, clozapine, 3rd generation OCP,
  erythropoietin, antidepressants, topical
  glucocorticoids, treatment for HIV

14
Lexchin and Bero conclusions

• Published research from drug companies is more
  likely to be favourable to the product
• Do companies selectively fund trials?
• Unlikely
• Is it of poorer quality?
• No
• Are inappropriate comparators chosen?
• Sometimes/often/a lot
• Is it publication bias?
• Yes

15
What is the evidence for it (3)?

• Melander et al BMJ 20033261171-3
• Question Is there selective reporting of
  sponsored studies by drug companies?
• Trials submitted to the Swedish drug regulatory
  authority (5 SSRIs, 42 trials)
• Multiple publication
• Selective publication
• Selective reporting

16
Melander et al conclusion

• Any attempt to recommend a specific selective
  serotonin reuptake inhibitor from the publicly
  available data ONLY is likely to be based on
  biased evidence.

17
What is the evidence for it (4)?

• Olson et al JAMA 20022872825-2828
• Question Is there publication bias in editorial
  decision making?
• 3 years, 745 manuscripts
• Positive vs negative OR 1.30 (0.87 to 1.86)
• Small effect of editorial decision making, much
  less than researchers not submitting negative
  studies
• Will this be true for journals less grand than
  JAMA?

18
What can be done about it (1)?

• Better conduct and reporting of RCTs (CONSORT)
• Better conduct and reporting of systematic
  reviews (QUORUM)
• Publication of unpublished trials
• Enlightened sponsors (a code of good practice
  Wager et al 2003 http//www.gpp-guidelines.org)
• Better editorial policies
• Vigilant editors and reviewers
• Responsible authors

19
What can be done about it (2)?

• Publication of original protocols and deviations
  from protocol
• Declaration of competing (financial) interests by
  authors, reviewers, and editors
• Declaration of sponsorship/funding
• Registering all clinical trials

20
How has the BMJ responded?

• A change in editorial thinking Is it the
  question that matters?
• It is
• Amnesty on unreported clinical trials
• More transparency (CONSORT, QUORUM, sponsorship,
  funding, competing interests)
• Theme issue on doctors and the drug industry
• ?Protocols
• ?Registering clinical trials

21
Conclusions

• Publication bias is an important problem that
  impacts on patient care
• There is much evidence to support its existence
• There are many players
• There are many ways to reduce its effect,
  examples of good practice
• Ultimately there is a big responsibility on
  sponsors of trials, authors, and editors