Randomized Controlled Trials: Allocation Concealment - PowerPoint PPT Presentation

Loading...

PPT – Randomized Controlled Trials: Allocation Concealment PowerPoint presentation | free to view - id: 575e2c-MzAyN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Randomized Controlled Trials: Allocation Concealment

Description:

Randomized Controlled Trials: Allocation Concealment Kenneth F. Schulz, PhD Fixation on Sequence Generation Traditionally, many medical researchers mistakenly ... – PowerPoint PPT presentation

Number of Views:186
Avg rating:3.0/5.0
Slides: 49
Provided by: Cynthi140
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Randomized Controlled Trials: Allocation Concealment


1
Randomized Controlled Trials Allocation
Concealment
  • Kenneth F. Schulz, PhD

2
Fixation on Sequence Generation
  • Traditionally, many medical researchers
    mistakenly consider the sequence generation
    process as randomization
  • They frequently ignore allocation concealment
  • Without adequate allocation concealment, proper
    random sequences can be subverted
  • For example, a properly generated sequence
    posted on a bulletin board

3
Allocation Concealment
  • Prevents selection bias by concealing the
    allocation sequence from those assigning
    participants to intervention groups
  • Until the moment of assignment

4
Mechanisms for Selection Biasin RCTs
  • If those responsible for entering participants
    know, or can detect, the upcoming treatment
    allocations
  • Decide entrance based on that knowledge
  • Channel participants with a better prognosis to
    the experimental group and those with a poorer
    prognosis to the control group, or vice versa
  • In any case, they introduce bias into the
    treatment comparison

5
Do Not Confuse Allocation Concealment with
Blinding
  • Allocation concealment seeks to prevent selection
    bias, protects assignment sequence before and
    until allocation, and can always be successfully
    implemented
  • In contrast, blinding seeks to prevent
    ascertainment bias, protects sequence after
    allocation, and cannot always be successfully
    implemented

6
Importance of Allocation Concealment
  • Unclearly concealed and inadequately concealed
    trials, compared to adequately concealed trials,
    exaggerated the estimates of an interventions
    effectiveness by 30 to 40, on average

Schulz KF, Chalmers I, Hayes RJ, Altman DG.
Empirical evidence of bias dimensions of
methodological quality associated with estimates
of treatment effects in controlled trials. JAMA
1995273408-412.
7
Similar Results on Allocation Concealment in a
Separate Study
  • Replicated our methods in different subject areas
    (digestive circulatory diseases and mental
    health)
  • Examined 11 meta-analyses
  • Inadequately concealed trials exaggerated
    estimates of effectiveness by 37

Moher D, et al. Does quality of reports of
randomised trials affect estimates of
intervention efficacy reported in meta-analyses?
Lancet 1998 352 609-13.
8
Allocation Concealment Summary
  • Adequate allocation concealment emerges from our
    analyses as crucial to reducing bias
  • Without it, the whole point of randomisation
    vanishes and bias may distort results
  • Our results support Mosteller and his colleagues

When the randomization leaks, the trials
guarantee of lack of bias runs down the drain
9
Method of allocation of treatment in 208
controlled trials in head injury Dickinson K, et
al., BMJ 20003201308-1311.
Method of allocation
No. of trials
Adequate
Centralised randomisation by telephone 1
Numbered/coded identical containers administered sequentially 11
Randomisation scheme controlled by pharmacy 8
Sequentially numbered, sealed, opaque envelopes 2

Not adequate
Other 18
Date of birth 1
Day of week 3
Alternation 3

Not stated 161
10
Actual allocation method with the method just
specified as random
  • Difficult (Peto and I. Chalmers experience)
  • MA of OCs
  • David G. knew the PI who put him in contact with
    the statistician
  • Described a great method . . . Post-it notes!
  • Seemed indignant . . . We asked for clarification
  • Perfect also for stratificationdifferent colors!!

11
Personal Accounts of Deciphering Assignment
Sequences
  • Conducted over 20 epidemiological workshops for
    medical residents and medical junior faculty
  • Each included 20-25 participants
  • Asked how many of the participants had
    deciphered, or had witnessed someone else
    decipher, an assignment sequence

12
Personal Accounts of Deciphering Assignment
Sequences (cont.)
  • With assurance of anonymity, more than half
    related at least one instance of deciphering
  • Not half of all trials
  • No accurate denominator
  • Nevertheless, not a rare occurrence

13
Decipherings Run the Gamut From Simple to
Intricate
  • Simple the most frequent and usually take
    advantage of inadequate allocation concealment
    schemes
  • Posting sequence on a bulletin board
  • Opening unsealed envelopes
  • Translucent envelopes to a light bulb
  • Opening many envelopes that were not sequentially
    numbered

14
Intricate Decipherings Less Frequent
  • Needed to circumvent more adequate allocation
    schemes
  • With sequentially numbered drug containers
  • Based on appearance of tablets in unsealed
    containers
  • Appearance of the label
  • With central allocation, obtaining the next few
    allocations at once
  • Envelopes to a hot light in radiology

15
Attempted to Decipher a Numbered Container Scheme
  • Gave up
  • Found attending physician rifling the P.I.s
    files for the assignment sequence
  • Horrified? No, impressed with his brilliance and
    proceeded to help
  • Should be kept in a locked location

16
Ignorance Is Not Bliss
At the annual meeting of the Society for
Clinical Trials, held in Houston, Schulz
reported data showing that the experts fears
about ignorance are justified. Schulz examined
250 reports of clinical trials and came up with
an intriguing finding
Rachel Nowak. Problems in clinical trials go far
beyond misconduct. Science 1994 264 1539.
17
Ignorance Is Not Bliss
Most physicians are not trained in basic
scientific principles, let alone clinical trials
Stephen George, Duke University, Chair of the
Statistics Committee for NCI, NIH Cancer Clinical
Cooperative Groups
I fantasize that in the future (training) will
become a requirement for running clinical trials
John Gallin, Director of NIHs Clinical Center
18
Rigorous Trials Annoy Humans
  • Investigators
  • Certain Ps to benefit
  • May want the results of study to reveal the
    truth
  • Proper trial procedures attempt to impede human
    inclinations
  • Hence, aspects of properly conducted RCTs annoy
    investigators

19
Without Methodological Rigour
  • The challenge of deciphering may frequently
    become too great a temptation to resist
  • Even without intent to bias
  • The only way to get rid of a temptation is to
    yield to it. Oscar Wilde
  • Deciphering may just reflect human
    inquisitiveness and ingenuity
  • Does not necessarily indicate scientific
    malevolence

20
Without Methodological Rigour
  • Or deciphering may reflect deliberate acts to
    alter findings
  • Whatever the motivations, innocent or naïve or
    deliberate, those actions undermine the validity
    of the trial
  • Investigators must devote diligent, persistent
    attention to randomization

21
Minimal Standards Common Allocation Concealment
Approaches
  • Sequentially numbered, opaque, sealed envelopes
    (SNOSE)
  • Pharmacy control
  • Numbered or coded containers
  • Central randomization
  • Realistically, these standards should be exceeded

22
Envelopes for Allocation Concealment
  • More susceptible to manipulation through human
    ingenuity
  • Less than ideal method of allocation concealment
  • If used, investigators must diligently develop
    and monitor the process

23
Envelopes (Cont.)
  • SNOSE
  • Ensure that the envelopes are opened sequentially
  • only after the Ps name and other details are
    written on the appropriate envelope

24
Envelopes (Contd)
  • Pressure-sensitive or carbon paper inside
  • Supports enrolment
  • Encourages orderly opening
  • Promotes proper assignment
  • Creates valuable audit trail
  • Cardboard or aluminum foil inside envelope

25
Pharmacy Controlled
  • Compliance with randomization/allocation
    concealment methods questionable
  • Aware of gross distortions
  • Pharmacy ran out of one drug . . .
  • Alternate assignment
  • Should not assume pharmacists knowledge
  • Ensure they, and all research partners, follow
    proper trial procedures

26
Numbered Containers
  • Excellent prevents foreknowledge
  • Must take proper precautions
  • Appearance exactly the same
  • Weight exactly the same
  • Sound when shaken exactly the same
  • Lettering exactly the same
  • Odor exactly the same
  • Sealed
  • Difference between concealment and blinding

27
Central Randomization
  • Frequently telephone
  • Fax
  • Email
  • Internet
  • Computer
  • Assignments only provided after participants
    irrevocably enrolled
  • Must check procedures for leaks
  • Monitor process for adherence

28
RCTs Anathema to the Human Spirit
  • Must acknowledge the vagaries of human nature
  • Must establish methodological safeguards that
    thwart attempts to contaminate trials with bias

29
Subversions of Sequences
  • Conscious or subconscious?
  • Direct or compensatory?
  • Doesnt matter
  • Biases the trial

30
Participants
Randomize
Placebo
New Oral Drug
60 Compliance
25 Non-compliance
75 Compliance
40 Non-compliance
Group representing the policy of no treatment
Group representing the policy of oral treatment
Outcome Outcome
31
RCT Compared the Effectiveness of Clofibrate in
Preventing Cardiac Deaths in Men Who Had Survived
a Myocardial Infarction
Clofibrate Placebo
5 Year mortality 20.2 20.9 (p .55)
Eliminating deviates from clofibrate (80 adherence) 15.0 20.9 (p lt .05)
Eliminating deviates from both groups 15.0 15.1
  • Authors state that
  • One can justify almost any conclusion, dependent
    upon the analysis chosen
  • Manipulating deviates leads to severe bias
  • Can you ever do so?

32
Exclusions After Randomization
  • Can introduce bias and should be carefully
    scrutinized
  • All randomized patients should be analyzed, and
    analyzed as part of the group to which they were
    initially assigned
  • ITT (Intention-to-treat)

33
Exclusions of LFU Damage Internal Validity
  • Without outcomes from those lost to follow-up,
    investigators have little choice but to exclude
    them from the analysis
  • Any losses damage internal validity
  • However, differential rates of loss among
    comparison groups cause major damage
  • Investigators must minimize their losses to
    follow-up

34
Retention of Trial Participants
  • Minimizing losses exudes difficulties
  • Eliminating losses may be impossible
  • But investigators too frequently profess
    insurmountable difficulties
  • Most should work harder to obtain higher
    follow-up rates
  • Investigators must commit adequate attention and
    resources to develop and implement procedures to
    minimize losses

35
Innovative Twists that Cultivate High Follow-up
Rates
  • Establish many conveniently placed FU facilities
  • Too often investigators expect participants to
  • visit a single, inconvenient location
  • Shortening data collection form caters to the
    participants wishes
  • Long sessions lead them to vote with their feet
  • Be considerate
  • Foster follow-up by not overburdening
  • Also may engender higher quality data on the
  • main items

36
END
37
Meta-analysis of Polyglycolic
Meta-analysis of polyglycolic acid versus catgut
for perineal repair on short term pain
1 adequately concealed trial OR .89 4
unclearly concealed trials OR .44
Protective effect of polyglycotic acid was 51
lower in the unclearly concealed trial
(ROR .49 95 CI of .35 - .69)
38
Meta-analysis of Corticosteriods
Meta-analysis of corticosteroids vs. none, after
preterm rupture of membranes, on respiratory
distress syndrome (RDS)
3 adequately concealed trials OR 0.72 4
unclearly concealed trials OR 0.53
Protective effect of corticosteroids was
exaggerated by 27 in the unclearly concealed
trials
ROR 0.73 95 CI of 0.35 - 1.50
39
Randomization Process (P 138B)
Proper approach to 4 OB/GYN Journals 4 General Medical Journals
Method of generation of random numbers 32 49
Randomization concealment 23 26
Both 9 15
  • In the 4 OB/GYN Journals, found the 2 treatment
    group sizes to be much too similar

Expected Observed
5 ( 5) 0
48 (50) 8
40
Separation of Executer and Generator
  • Person(s) who prepare the randomization scheme
    should not determine eligibility, administer
    treatment, or assess outcome
  • Neglected critical element
  • Obviously importantregardless of methodological
    quality, access to sequence and thus opportunity
    to introduce bias
  • Faults in this element may be the crack though
    which much of the bias seeps into controlled
    trials

41
Not quite as random as I pretended
Martyn C. Not quite as random as I pretended.
Lancet 1996 34770.
Bias in the way treatments are assigned can be
a more powerful determinant of the outcome of the
trial than the treatments that are being
investigated.
Trialists often find it hard to prevent their
hopes that a new treatment might prove to be
beneficial from evolving into a belief that it
really is so.
42
RCT in Norway
  • Examined the effect of Chlorhexidine compared to
    placebo on postpartum infection and Strep B
    infections in neonates
  • Reported as a randomized controlled trial in the
    published article

Insert published article
43
RCT in Norway
  • Midwives knew on which shelf the Chlorhexidine
    and placebo were kept in the cabinet and just
    proceeded to administer whatever they deemed
    appropriate
  • Even if they forgot the shelf, they could smell
    the difference
  • No monitoring of the allocation process

44
Letter from a Physical Therapist
  • A friend of mine conducted an RCT comparing 2
    training regimens following surgical repair of
    the knee for anterior cruciate ligament damage
  • randomization method used was alternating
    assignment
  • Referred to one patient as difficult and asked
    the friend to which group she was assigned
  • Friend responded that she was in the group that
    required slightly less contact and that her
    inclusion was by his choice

45
Not so blind, after all
  • Sometimes subverting can work against a
    treatments apparent effectiveness

Medical staff may try to help their sickest
patients by sneaking them into the treatment
group instead of the control group.
The drug would then have to be significantly
better than conventional treatment just to appear
equal in efficacy
Wallich P. Not so blind after all. Scientific
American May 1996.
46
RCTs Anathema to the Human Spirit
  • Scientific need to obtain unbiased data from an
    inherently biased source human beings
  • Many trialists are unaware of the rationale
    behind RCTs
  • And the need for trial rigor
  • Some may intellectually grasp the need, but once
    immersed in a trial, have many contradictory
    interests (e.g. episiotomy trial in Canada)

47
Anathema A Psychologists View
  • Fundamental conflict between scientific and
    human imperatives Many probably think they
    already know what treatment is best.
  • Geoffrey Scobie, psychologist at the University
    of Glasgow Doctors cheat because they know
    whats best
  • New Scientist, 16 December 95, p. 10
  • Of course, what we think we know is frequently
    wrong

48
Reporting of Exclusions, Double-blinding, and
Schedule Generation Related to the Level of
Allocation Concealment for 250 Trials
Authors Reported Adequately Concealed (n79) Unclearly Concealed (n150) Inadequately Concealed (n21)
No Exclusions (Apparent) 53 67 76
Double-blinding 73 39 14
Adequate generation of schedule 29 15 0
About PowerShow.com