To Prevent Selection Bias - PowerPoint PPT Presentation


Title: To Prevent Selection Bias


1
To Prevent Selection Bias
Minimal Balance is Sufficient
  • Wenle Zhao, PhD
  • Medical University of South Carolina, Charleston,
    SC, 29425, USA
  • Society for Clinical Trials 36th Annual Meeting
  • Arlington, VA, USA - May 17-20, 2015

2
Contents
  1. Where does Selection Bias Come From?
  2. How to prevent selection bias?
  3. How to avoid random serious imbalance?

3
The Worst Thing in the World of Clinical Trials
Funding?
Recruitment?
A Completed Trial with Suspicious Selection Bias.
4
Defense Measurements against Selection Bias
Subject Enrollment
Treatment Allocation
Outcome Assessment
Real-time Subject Randomization
Treatment Masking
Random Allocation
Allocation Concealment
The only reliable protection left against
selection bias.
5
Allocation Randomness
Target allocation ratio
Random variable U(0,1)
To balance treatment distribution Permuted Block
Randomization Biased Coin, Urn Design
To balance baseline covariate Stratified
Randomization Minimization
Allocation Randomness
Complete Randomization
Permuted Block Randomization
Minimization
6
Predictability Defeats Concealment Masking
Deterministic Assignment
100
87
Proportion of DA
50
33
25
20
0
B 2
B 4
B 6
B 8
Minimization
Permuted Block Randomization
7
Evidence of Selection Bias in Randomized Trials
1. Heparin for myocardial infarction
E
2. University Group Diabetes Program
E
3. Talc and mustine for pleural effusions
P
4. Tonsillectomy for recurrent throat infection
in children
P
5. Oxytocin and amniotomy for induction of labor
P
6. Western Washington Intracoronary Streptokinese
Trial
P
7. RSV immune globulin in infants and young
children
E
8. A trial to assess episiotomy
E
9. Canadian National Breast Cancer Screening Study
Selection bias evidence identified
P
E
10. Surgical Trial
P
11. Lifestyle Heart Trial
P
12. Coronary Artery Surgery Trial
E
13. Etanercept for children with juvenile
rheumatoid arthritis
P
14. Edinburgh Randomized Trial of Breast Cancer
Screening
E
15. Captopril Prevention Project
P
16. Göteborg (Swedish) Mammography Trial
P
17. HIP Mammography Trial
E
18. Hypertension Detection and Follow-up Program
E
19. Randomized Trial to prevent vertical
transmission of HIV-1
P
20. Effectiveness trial of diagnostic test
P
21. S African trial of high-dose chemotherapy for
metastatic breast cancer
E
22. Randomized study of a culturally sensitive
AIDS education program
P
23. Runaway Youth Study
P
24. Cluster randomized trial of palliative care
P
Suspicious election bias due to p-value lt 0.05
P
25. Randomized trial of methadone with or without
heroin
P
26. Randomized NINDS trial of tissue plasminogen
activator for acute ischemic stroke
P
27. Norwegian Timolol Trial
P
28. Laparoscopic versus open appendectomy
P
29. The Losartan Intervention for Endpoint
reduction in Hypertension Study
P
30. The Heart Outcomes Prevention Evaluation
Study
P
8
Protect Trials Against Selection Bias
Selection bias will result small
p-values Complete randomization may (5 chance)
see a p-value lt 0.05
P lt 0.05 ?
9
The Logic
  • Real-time complete randomization
  • Eliminates selection bias due to allocation
    predictability
  • Eliminates selection bias due to allocation
    concealment failure
  • Totally eliminates selection bias
  • Without selection bias, complete randomization
    may still have
  • Imbalance in treatment distribution
  • ? Power loss is trivial
  • Imbalance is baseline covariate distribution
  • ? Adjustment, not balancing, is the solution
  • Serious baseline covariate imbalance with p-value
    lt 0.05
  • ? 5 chance for any covariate
  • ? 60 chance for at least one in 10 covariates
  • ? Suspicion of selection bias
  • ? Trouble in trial result interpretation

10
Options We Have
  • Stratified Restricted Randomization
  • Permuted Block Randomization
  • Biased Coin Design - Efron
  • Urn Design - Wei
  • Big Stick Design Soares Wu
  • Maximal Procedure Berger et al.
  • Block Urn Design Zhao Weng
  • Unnecessarily tighten control imbalances.
  • Disabled when number of strata getting large.
  • Minimization
  • Most assignments are deterministic.
  • Dynamic Hierarchy Balancing
  • Hierarchy order is hard to justify.

11
Minimal Sufficient Balance Procedure
Subject ready for randomization
T-test for continuous var. ?2 test for
categorical var.
N
Any serious imbalance?
Any p-value lt 0.2?
Y
Current assignment can effectively reduce
imbalances?
Y
N
Complete randomization
Biased coin assignment
The proportion depends on p-value threshold and
biased coin probability
Next subject
12
Example NINDS rt-PA Stroke Study
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
Serious imbalances found in 2 of the 11 baseline
covariates.
13
Example NINDS rt-PA Stroke Study
NINDS rt-PA Stroke study data. Simulations
5000
  • Baseline covariates
  • Severity (NIHSS)
  • Age
  • Onset to treat
  • Glucose
  • Center

14
Example NINDS rt-PA Stroke Study
Balance 11 baseline covariates
Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario.
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Low 2.5 boundary 0.226 0.259 0.237 0.262 0.244 0.262 0.214 0.245 0.239 0.252 0.246
Low 5 boundary 0.276 0.295 0.279 0.288 0.280 0.292 0.277 0.288 0.278 0.281 0.287
Low 10 boundary 0.309 0.341 0.316 0.323 0.315 0.326 0.309 0.322 0.319 0.322 0.330
Median 0.605 0.631 0.626 0.624 0.624 0.638 0.609 0.634 0.626 0.638 0.610
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
15
Summary
  • Complete randomization eliminates selection bias
    due to allocation predictability.
  • Real-time randomization eliminates selection bias
    due to allocation concealment failures.
  • Minimization method has the highest proportion of
    deterministic assignments, and therefore is
    vulnerable to selection bias.
  • Power loss due to treatment imbalance is trivial.
  • Justification, not balancing, is the solution for
    covariate confounding effects.
  • Using Minimal Sufficient Balancing to prevent
    random serious imbalances, while maintaining a
    high level of allocation randomness.

16
Thank You! Contact me at zhaow_at_musc.edu
17
Some of my works on Randomization
  • Zhao W, Ciolino J, Palesch Y. Step-forward
    randomization in multicenter emergency treatment
    clinical trials. Acad Emerg Med. 2010
    Jun17(6)659-65. doi 10.1111/j.1553-2712.2010.00
    746.x. PMID 20624149

.
  • Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative
    comparison of randomization designs in sequential
    clinical trials based on treatment balance and
    allocation randomness. Pharm Stat. 2012
    Jan-Feb11(1)39-48. doi 10.1002/pst.493. PMID
    21544929
  • Zhao W, Weng Y. Block urn design - a new
    randomization algorithm for sequential trials
    with two or more treatments and balanced or
    unbalanced allocation. Contemp Clin Trials. 2011
    Nov32(6)953-61. doi 10.1016/j.cct.2011.08.004.
    PMID 21893215
  • Zhao W, Hill MD, Palesch Y. Minimal sufficient
    balance--a new strategy to balance baseline
    covariates and preserve randomness of treatment
    allocation. Stat Methods Med Res. 2012 Jan 26.
    Epub ahead of print PMID 22287602
  • Zhao W. Selection bias, allocation concealment
    and randomization design in clinical trials.
    Contemp Clin Trials. 2013 Sep36(1)263-5. doi
    10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No
    abstract available. PMID 23871796
  • Zhao W. A better alternative to stratified
    permuted block design for subject randomization
    in clinical trials. Stat Med. 2014 Dec
    3033(30)5239-48. doi 10.1002/sim.6266. PMID
    25043719
  • Zhao W, Durkalski V. Managing competing demands
    in the implementation of response-adaptive
    randomization in a large multicenter phase III
    acute stroke trial. Stat Med. 2014 Oct
    1533(23)4043-52. doi 10.1002/sim.6213. Epub
    2014 May 22. PMID 24849843
  • Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of
    statistical and operational properties of subject
    randomization procedures for large multicenter
    clinical trial treating medical emergencies.
    Contemp Clin Trials. 2015 Mar41211-8. doi
    10.1016/j.cct.2015.01.013. Epub 2015 Jan 29.
    PMID 25638754
View by Category
About This Presentation
Title:

To Prevent Selection Bias

Description:

To Prevent Selection Bias. Wenle Zhao, PhD. Medical University of South Carolina, Charleston, SC, 29425, USA. Society for Clinical Trials 36th Annual Meeting – PowerPoint PPT presentation

Number of Views:34
Avg rating:3.0/5.0
Slides: 18
Provided by: sct85
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: To Prevent Selection Bias


1
To Prevent Selection Bias
Minimal Balance is Sufficient
  • Wenle Zhao, PhD
  • Medical University of South Carolina, Charleston,
    SC, 29425, USA
  • Society for Clinical Trials 36th Annual Meeting
  • Arlington, VA, USA - May 17-20, 2015

2
Contents
  1. Where does Selection Bias Come From?
  2. How to prevent selection bias?
  3. How to avoid random serious imbalance?

3
The Worst Thing in the World of Clinical Trials
Funding?
Recruitment?
A Completed Trial with Suspicious Selection Bias.
4
Defense Measurements against Selection Bias
Subject Enrollment
Treatment Allocation
Outcome Assessment
Real-time Subject Randomization
Treatment Masking
Random Allocation
Allocation Concealment
The only reliable protection left against
selection bias.
5
Allocation Randomness
Target allocation ratio
Random variable U(0,1)
To balance treatment distribution Permuted Block
Randomization Biased Coin, Urn Design
To balance baseline covariate Stratified
Randomization Minimization
Allocation Randomness
Complete Randomization
Permuted Block Randomization
Minimization
6
Predictability Defeats Concealment Masking
Deterministic Assignment
100
87
Proportion of DA
50
33
25
20
0
B 2
B 4
B 6
B 8
Minimization
Permuted Block Randomization
7
Evidence of Selection Bias in Randomized Trials
1. Heparin for myocardial infarction
E
2. University Group Diabetes Program
E
3. Talc and mustine for pleural effusions
P
4. Tonsillectomy for recurrent throat infection
in children
P
5. Oxytocin and amniotomy for induction of labor
P
6. Western Washington Intracoronary Streptokinese
Trial
P
7. RSV immune globulin in infants and young
children
E
8. A trial to assess episiotomy
E
9. Canadian National Breast Cancer Screening Study
Selection bias evidence identified
P
E
10. Surgical Trial
P
11. Lifestyle Heart Trial
P
12. Coronary Artery Surgery Trial
E
13. Etanercept for children with juvenile
rheumatoid arthritis
P
14. Edinburgh Randomized Trial of Breast Cancer
Screening
E
15. Captopril Prevention Project
P
16. Göteborg (Swedish) Mammography Trial
P
17. HIP Mammography Trial
E
18. Hypertension Detection and Follow-up Program
E
19. Randomized Trial to prevent vertical
transmission of HIV-1
P
20. Effectiveness trial of diagnostic test
P
21. S African trial of high-dose chemotherapy for
metastatic breast cancer
E
22. Randomized study of a culturally sensitive
AIDS education program
P
23. Runaway Youth Study
P
24. Cluster randomized trial of palliative care
P
Suspicious election bias due to p-value lt 0.05
P
25. Randomized trial of methadone with or without
heroin
P
26. Randomized NINDS trial of tissue plasminogen
activator for acute ischemic stroke
P
27. Norwegian Timolol Trial
P
28. Laparoscopic versus open appendectomy
P
29. The Losartan Intervention for Endpoint
reduction in Hypertension Study
P
30. The Heart Outcomes Prevention Evaluation
Study
P
8
Protect Trials Against Selection Bias
Selection bias will result small
p-values Complete randomization may (5 chance)
see a p-value lt 0.05
P lt 0.05 ?
9
The Logic
  • Real-time complete randomization
  • Eliminates selection bias due to allocation
    predictability
  • Eliminates selection bias due to allocation
    concealment failure
  • Totally eliminates selection bias
  • Without selection bias, complete randomization
    may still have
  • Imbalance in treatment distribution
  • ? Power loss is trivial
  • Imbalance is baseline covariate distribution
  • ? Adjustment, not balancing, is the solution
  • Serious baseline covariate imbalance with p-value
    lt 0.05
  • ? 5 chance for any covariate
  • ? 60 chance for at least one in 10 covariates
  • ? Suspicion of selection bias
  • ? Trouble in trial result interpretation

10
Options We Have
  • Stratified Restricted Randomization
  • Permuted Block Randomization
  • Biased Coin Design - Efron
  • Urn Design - Wei
  • Big Stick Design Soares Wu
  • Maximal Procedure Berger et al.
  • Block Urn Design Zhao Weng
  • Unnecessarily tighten control imbalances.
  • Disabled when number of strata getting large.
  • Minimization
  • Most assignments are deterministic.
  • Dynamic Hierarchy Balancing
  • Hierarchy order is hard to justify.

11
Minimal Sufficient Balance Procedure
Subject ready for randomization
T-test for continuous var. ?2 test for
categorical var.
N
Any serious imbalance?
Any p-value lt 0.2?
Y
Current assignment can effectively reduce
imbalances?
Y
N
Complete randomization
Biased coin assignment
The proportion depends on p-value threshold and
biased coin probability
Next subject
12
Example NINDS rt-PA Stroke Study
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
Serious imbalances found in 2 of the 11 baseline
covariates.
13
Example NINDS rt-PA Stroke Study
NINDS rt-PA Stroke study data. Simulations
5000
  • Baseline covariates
  • Severity (NIHSS)
  • Age
  • Onset to treat
  • Glucose
  • Center

14
Example NINDS rt-PA Stroke Study
Balance 11 baseline covariates
Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario.
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Low 2.5 boundary 0.226 0.259 0.237 0.262 0.244 0.262 0.214 0.245 0.239 0.252 0.246
Low 5 boundary 0.276 0.295 0.279 0.288 0.280 0.292 0.277 0.288 0.278 0.281 0.287
Low 10 boundary 0.309 0.341 0.316 0.323 0.315 0.326 0.309 0.322 0.319 0.322 0.330
Median 0.605 0.631 0.626 0.624 0.624 0.638 0.609 0.634 0.626 0.638 0.610
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
15
Summary
  • Complete randomization eliminates selection bias
    due to allocation predictability.
  • Real-time randomization eliminates selection bias
    due to allocation concealment failures.
  • Minimization method has the highest proportion of
    deterministic assignments, and therefore is
    vulnerable to selection bias.
  • Power loss due to treatment imbalance is trivial.
  • Justification, not balancing, is the solution for
    covariate confounding effects.
  • Using Minimal Sufficient Balancing to prevent
    random serious imbalances, while maintaining a
    high level of allocation randomness.

16
Thank You! Contact me at zhaow_at_musc.edu
17
Some of my works on Randomization
  • Zhao W, Ciolino J, Palesch Y. Step-forward
    randomization in multicenter emergency treatment
    clinical trials. Acad Emerg Med. 2010
    Jun17(6)659-65. doi 10.1111/j.1553-2712.2010.00
    746.x. PMID 20624149

.
  • Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative
    comparison of randomization designs in sequential
    clinical trials based on treatment balance and
    allocation randomness. Pharm Stat. 2012
    Jan-Feb11(1)39-48. doi 10.1002/pst.493. PMID
    21544929
  • Zhao W, Weng Y. Block urn design - a new
    randomization algorithm for sequential trials
    with two or more treatments and balanced or
    unbalanced allocation. Contemp Clin Trials. 2011
    Nov32(6)953-61. doi 10.1016/j.cct.2011.08.004.
    PMID 21893215
  • Zhao W, Hill MD, Palesch Y. Minimal sufficient
    balance--a new strategy to balance baseline
    covariates and preserve randomness of treatment
    allocation. Stat Methods Med Res. 2012 Jan 26.
    Epub ahead of print PMID 22287602
  • Zhao W. Selection bias, allocation concealment
    and randomization design in clinical trials.
    Contemp Clin Trials. 2013 Sep36(1)263-5. doi
    10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No
    abstract available. PMID 23871796
  • Zhao W. A better alternative to stratified
    permuted block design for subject randomization
    in clinical trials. Stat Med. 2014 Dec
    3033(30)5239-48. doi 10.1002/sim.6266. PMID
    25043719
  • Zhao W, Durkalski V. Managing competing demands
    in the implementation of response-adaptive
    randomization in a large multicenter phase III
    acute stroke trial. Stat Med. 2014 Oct
    1533(23)4043-52. doi 10.1002/sim.6213. Epub
    2014 May 22. PMID 24849843
  • Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of
    statistical and operational properties of subject
    randomization procedures for large multicenter
    clinical trial treating medical emergencies.
    Contemp Clin Trials. 2015 Mar41211-8. doi
    10.1016/j.cct.2015.01.013. Epub 2015 Jan 29.
    PMID 25638754
About PowerShow.com