Top Trial Cheats

1
Top Trial Cheats

• Dr Rod Stables
• The Cardiothoracic Centre
• Liverpool UK
• No Conflicts of Interest to Declare

2
Presentation Outline

• Examples without accusation
• Three topic areas
• Establishing the denominators
• Lost or not evaluated at follow-up
• Odds ratio v Risk ratios
• Composite outcome measures
• Mixing safety and efficacy measures

3
Identify the Denominator Tracking Trial Subjects

• Need to follow the fate of ALL patients
• From inclusion to outcomes
• Warning signs !
• Results expressed as simple age
• Smokescreen phrases
• In patients with outcome data
• In evaluable (surviving) patients
• When assessed at follow-up

4
Identify the Denominator Examples

• Rickards and Stables CABRI long term follow-up
• Widely presented (but never published)
• Every result presented as a value
• Masked incomplete follow-up
• No data on gt120 Dutch patients
• More sinister impact - reported outcome measures

5
Not Assessed for the Outcome Measure ?

• Trial of drug A - exercise capacity in heart
  failure
• 100 patients randomised 1 1 drug v placebo
• Outcome measure
• Mean exercise duration at FUP
• Baseline exercise duration - equal
• Follow up Drug 3.2 mins Placebo 2.4 mins
• Clinical and statistical significance
• Mechanism ?

6
Not Assessed for the Outcome Measure ?

• Widespread implications
• Incomplete QCA follow up in angiographic trials
• Numerical handling of extreme examples
• QCA - MLD of occluded vessel ? 0
• QoL - score of dead patient 0 (?excluded)
• Competing outcomes
• ? mortality ? repeat revascularisation
• Event curve analysis - No of patients at risk

7
SoS Trial Repeat Revasc Event Curve
8
BASIL Trial Amputation - free Survival
9
Solutions ?

• Clear exposition of patient flows

10
Trial Profile and Patient Flow Diagram
11
Solutions ?

• Clear exposition of patient flows
• All event rates expressed n / d (p) (CIs)
• Describe status of patients not assessed for
  outcome measures
• Express rates with correct denominators eg
• Repeat revasc rate per 100 patient years of Fup
• Full description of complex data handling methods
• QCA QoL etc

12
Odds and Risk
13
Odds and Risk

• Throw one of a pair of dice - Aim - To get a
  6
• Odds against 51 (Odds)
• Event occurs 16 of all throws (Risk)
• Scope for confusion and manipulation of data
• . and audience
• Most observers interpret odds ratio as relative
  risk

14
Example Relative Risk in a Classic RCT

• Randomised trial Tx A v Tx B
• Each group comprises 100 patients
• Death rate on Tx A 50/100 ( 0.5)
• Death rate on Tx B 25/100 ( 0.25)
• Relative risk 0.5 / 0.25 2
• (p values and confidence intervals)

15
Odds Ratio Calculation

• Tx A No dead 50 (a) No alive 50 (c)
• Tx B No dead 25 (b) No alive 75 (d)
• Odds ratio ad/bc (50 x 75)/(25 x 50)
• 3
• Sounds more !!

16
From the Thesis of Dr R H Stables DM (Oxon) ..
17
Relative Risk and Odds Ratio

• If event rate is low OR is approx RR
• If OR is interpreted as RR it will overstate the
  risk (benefit)

Reported Ratio
Event Rate
18
Some Basic Rules

• Odds ratio correct in 2 x settings
• Backward investigation - Case control

19
Case Control Studies

• Breast cancer patients - series of 200
• History of OCP 50 (a) No OCP 150 (c)
• Odds a/c 0.33
• Age sex matched controls - series of 200
• History of OCP 25 (b) No OCP 175 (d)
• Odds b/d 0.14
• Odds Ratio (a/c) / (b/d) a/c x d/b
• Odds ratio ad/bc (50 x 175)/(150 x 25) 2.3

20
Some Basic Rules

• Odds ratio correct in 2 x settings
• Backward investigation - Case control
• Multiple regression analyses
• Report independent association strength
• factor and outcome
• In all others - RCT and cohort studies
• Risk ratio applicable and best

21
Mixing Efficacy and Safety Outcomes
22
Mixing Efficacy and Safety Outcomes

• Developing area - Subject of continued debate
• CVA addition to MACE in revascularisation trials
• PCI trials - MACE outcomes
• ? Add in bleeding complications
• Usually PCI related (puncture adjunct drugs)
• Seems reasonable ?
• Value in trials of antithrombotic therapy ?

23
Mixing Efficacy and Safety Outcomes

• Cloud and confuse
• Data analysis and reporting
• Procedural development
• BENESTENT 1
• Stent puncture site complications 7 !!
• Unrelated to coronary efficacy of stent
• Reduced to current levels - changes in
• Drugs devices techniques - experience

24
Mixing Efficacy and Safety Outcomes

• Efficacy and safety issues
• Parallel tracks - causation and solution
• Further complicate analysis and reporting
• Minefield of composite outcomes
• Perhaps best avoided

25
Questions and Discussion