Title: Sample Size And Power Warren Browner and Stephen Hulley
1Sample Size And PowerWarren Browner and Stephen
Hulley
- The ingredients for sample size planning, and how
to design them - An example, with strategies for minimizing sample
size
2Sampling and Inference
- A sample is designed to represent a larger
population - Therefore, findings in the sample allow
inferences about events in the population - Problem what if the inferences are wrong?
- Finding something in the sample that isnt real
in the population - Missing something that is real
3Preventing Wrong Inferences
- Difficult when caused by systematic error (bias)
- Easy when caused by Random error (chance)
- Solution increase sample size
- Problem cost, feasibility
- Goldilocks solution a sample size that is big
enough but not too big
4Ingredients For Planning Sample Size in an
Analytic Study or RCT
- Hypothesis
- Null and alternative
- One-sided vs two-sided
- Statistical test
- Type of variables
- Effect size (and its variance)
- Power and alpha
5Research Hypothesis
- A clear statement of what you are studying
- Simple one predictor, one outcome
- Specific who, what, when, where
- Stated in advance
6Research Hypothesis
- In patients with early ALS seen at UCSF in 2007,
those randomly assigned to be treated with newmol
will have a lower 1-year mortality than those
randomly assigned to placebo.
7The Null Hypothesis
- Theres nothing going on.
- Purpose in life to be rejected in favor of its
alternative. - In patients with early ALS seen at UCSF in 2007,
those randomly assigned to be treated with newmol
will have the same 1-year mortality as those
randomly assigned to placebo.
8Whats This All About?
- A long time ago, statisticians figured out the
probability that a sample of a given size would
find something even if there were nothing going
on in the population.
9This means that...
- After a study, we can determine the likelihood
that whatever we found in our sample could have
occurred by chance... - Even if nothing was going on in the population
(i.e., the null hypothesis was true)--a Type I
error - If this is very unlikely (say lt 1 in 20) we
reject the null hypothesis in favor of the
alternative hypothesis we call the finding
statistically significant (P lt .05)
10Two-sided Alternative Hypothesis
- In patients with early ALS seen at UCSF in 2007,
those randomly assigned to be treated with newmol
will have a different 1-year mortality than those
randomly assigned to placebo.
11Two One-sided Alternative Hypotheses
- Side A In patients with early ALS seen at UCSF
in 2007, those randomly assigned to be treated
with newmol will have a higher 1-year mortality
than those randomly assigned to placebo. - Side B In patients with early ALS seen at UCSF
in 2007, those randomly assigned to be treated
with newmol will have a lower 1-year mortality
than those randomly assigned to placebo.
12If The Null Hypothesis Is True
- By chance alone, each of the two one-sided
alternative hypotheses is... - Possible
- Equally likely
- Wrong
- Thus a two-sided alternative hypothesis has twice
the likelihood of happening by chance alone
13Next Ingredient Statistical Test (Types of
Variable)
- The statistical test determines how the sample
size will be calculated - The type of predictor and outcome variable
determine which statistical test will be used to
analyze the data - Both dichotomous Chi square
- One dichotomous, one continuous t test
- Both continuous correlation coeff or t test
14Statistical Test (Types of Variable)
- ALS study
- Predictor newmol vs placebo
- Outcome dead
- Both are dichotomous
- Chi square test
15Next IngredientEffect Sizes (dichotomous
variables)
- How big an effect you anticipate seeing
- Newmol halves mortality
- newmol 5, Placebo 10
16Penultimate Ingredient Power
- The chance of finding something in your sample if
its really going on in the population (avoiding
a Type II error) - Something the effect size (or greater)
- Usually set at 80 or 90
- (1 - beta)
17and the Final Ingredient Alpha
- The chance of finding something in your sample if
theres nothing going on in the population.
18Alpha Explained
- The level of statistical significance (ie, the
p-value that will be considered significant) - The pre-set maximum chance of finding something,
if it really isnt there. - Usually set at 0.05.
- May be one-sided or two-sided.
19Sidedness Of Alpha
- With a two-sided alternative hypothesis, you have
two chances of finding something that isnt
really there - One (equal) chance for each side.
- So a one-sided alpha of 0.05 corresponds to a
two-sided alpha of 0.10.
20SAMPLE SIZE AN EXAMPLE
- Null hypothesis
- In patients with early ALS seen at UCSF in 2007,
those randomly assigned to be treated with newmol
will have the same 1-year mortality as those
randomly assigned to placebo. - Two-sided alternative hypothesis
- Dichotomous predictor and outcome
- Effect size 10 mortality 5
- Power, alpha 90, 0.05 (two-sided)
21THE SAMPLE SIZE IS
- Appendix 6.B
- Smaller of P1 and P2 0.05 power of 90 alpha
of 0.05 (two-sided) - Difference 0.05
- 381
- 473
- 620
- This is per group
22Sample Size Reduction Strategy 1Statistical
Manipulation
- Use a lower power
- Use a one-sided alpha
- Power of 80
- One-sided alpha of 0.05
23The New Sample Size Is
- Appendix 6.B
- Smaller of P1 and P2 0.05 power of 80 alpha
of 0.05 (one-sided) - Difference 0.05
- 381
- 473
- 620
- This is also per group
24SS Reduction Strategy 2 Use A More Common
Outcome
- Change from 1-year mortality to 2-year mortality
or loss of independent living - Placebo 40
- Newmol 20
25The New Sample Size Is
- Appendix 6.B
- Smaller of P1 and P2 0.20 power of 80 alpha
of 0.05 (two-sided) - Difference 0.20
- 74
- 91
- 118
26SS Reduction Strategy 3 Use A Continuous
Outcome
- Change mortality or loss of independent living
to muscle strength - NOTE Big change in research question and
research hypothesis. - New null hypothesis
- In patients with early ALS seen at UCSF in 2007,
those randomly assigned to be treated with newmol
will have the same grip strength at the end of
six months as those treated with placebo. - Two-sided alternative hypothesis
27Estimate The Mean And Variability Of Grip Strength
- Patients with untreated ALS have a (mean SD)
grip strength of 20 10 kg after 6 months of
disease - Newmol may improve that by 25
28Then
- Grip strength
- Placebo 20 kg
- Newmol 25 kg (25 more)
- Effect size 5 kg SD 10 kg
- Standardized effect size E/S 5/10 0.5
29The New Sample Size Is ...
- Appendix 6.A
- E/S 0.5
- ß 0.20, Alpha (two-sided) 0.05
- N 64 per group
30Ss Reduction Strategy 4 Use A More Precise
Outcome
- Buy a better instrument to measure grip strength
- Use a well-defined protocol
- Repeat measurements on two consecutive days
- Reduce SD from 10 kg to 8 kg
31The New Sample Size Is ...
- New E/S 5 kg/8 kg 0.625
- ß 0.20, Alpha (two-sided) 0.05
- N about 45 per group
- This helped quite a bit.
32SS Reduction Strategy 5 Use Paired Measurements
- Most of the variability in grip strength at the
end of the study is likely to be due to
differences between subjects in grip strength at
the beginning of the study. - Switch the outcome to change in grip strength
from the beginning to the end of the study.
33Paired Measurements
- Each subject contributes a pair of measurements
(before, after) - The outcome variable is the difference between
that pair for each subject. - The SD of the change in a measurement is usually
lt than the SD of the measurement - SD of change in grip strength is 5 kg
- New standardized effect size 5/5 1.0
34The New Sample Size Is...
- E/S 1.0
- ß 0.20, Alpha (two-sided) 0.05
- N 17 per group
- We now have a potentially do-able study, albeit
one that is very different from the original aim.
35The Bottom Line
- Sample size estimation is an integral part of
study planning - Almost never the last thing you do
- More often, one of your first tasks
36SAMPLE SIZE PLANNING REVIEW OF INGREDIENTS
- Looking for something in a sample
- Hypotheses (null and alternative)
- Will you be able to ...
- Know its there in the population if you find it
in your sample (avoid a Type I error) - Test of significance, alpha
- Find it in your sample if its there in the
population (avoid a type II error)? - Effect size, power