Basic statistics

1
Is bigger better?   An introduction to sample
size calculations       Presented by Dr Adrian
Esterman
2
Scenario 2 Power
Scenario 1 Precision
All studies
Hypothesis testing
Descriptive
Sample surveys Quality control
Simple - 2 groups
Complex studies
3
Scenario 1
Suppose we want to estimate the proportion of
people in our target population with a given
characteristic

• The proportion with depression
• The proportion with an artficial leg
• The proportion receiving incorrect medication

4
Scenario 1
Example

• My target population is all South Australians
  aged 17 and over
• I want to find out what proportion have an
  undergraduate degree
• Please raise your hand if you have an
  undergraduate degree

5
Scenario 1
Random
Target Population
Sample
Measure Characteristic
Infer
6
Scenario 1
True proportion in target population
P Estimated proportion from sample p How
likely is it that p is exactly equal to P?
7
Scenario 1
We would like 95 times out of 100, P to fall in
this range
0
1
p
Sample
8
Scenario 1
The range of plausible values of our sample
proportion p in which the true population
proportion P is likely to fall 95 times out of
100 is called the 95 Confidence Interval for P
9
Scenario 1
95 CI for P
0
1
p
Sample
10
Scenario 1
The 95 CI for p is a measure of how accurate
your sample estimate is of the true population
proportion
95 Confidence Interval
Sample size
11
Scenario 1
Example We want to estimate the proportion of the
South Australian population with COPD. We think
it will be about 12.
We would like a 95 CI of p 2.
12
Scenario 1
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
Statcalc
Statcalc is included as part of the Epiinfo suite
of programs. This is available free of charge
from http//www.cdc.gov/epiinfo/
21
Scenario 2
We wish to formally test the difference between
two means or two proportions
22
Scenario 2
Three bits of information required to determine
the sample size
Type I II errors
Variation
Clinical effect
23
Type I II errors
Process of hypothesis testing

1. State a Null hypothesis (H0)
2. State an Alternative hypothesis (HA)
3. Decide on a suitable statistical test based on
   the Null hypothesis
4. Calculate the test statistic
5. Check the associated probability (p-value)
6. If p ? 0.05 reject the Null hypothesis

24
Type I II errors
Process of hypothesis testing

• Note
• If the Alternative hypothesis is
• parameter 1 ? parameter 2
• we calculate the p-value for a two-sided test
• If the Alternative hypothesis is
• parameter 1 gt parameter 2
• we calculate the p-value for a one-sided test

25
What is a p-value?
Type I II errors

• 1. It is a probability, and hence lies between 0
  and 1.
• 2. It is a measure of surprise. In fact how
  surprised we are to get a test statistics that
  large, if the Null hypothesis were true.

26
Type I II errors
Type I and II errors
Statistical True state of null
hypothesis decision Hypothesis true
Hypothesis false Reject Null Type I error
Correct (Power) hypothesis Accept Null
Correct Type II error hypothesis
27
Type I II errors
What causes a Type I error

• Bias
• Confounding
• Effect modification
• Misclassification

28
Type I II errors
What causes a Type II error

• Sample size too small
• Confounding
• Effect modification
• Misclassification

29
Type I II errors
Example of setting error levels

• New drug for lowering cholesterol
• Slightly better efficacy than existing drugs
• Much more expensive than existing drugs

What are the consequences of making a Type I
error? What are the consequences of making a Type
II error?
30
Type I II errors
Example 1

• New drug for lowering cholesterol
• Slightly better efficacy than existing drugs
• Much more expensive than existing drugs

• Conclusion
• Requires stringent Type I error (say 0.01)
• Can managed with relaxed Type II error (say
  0.20)

31
Type I II errors
Example 2

• Trial of new brochure to help people quit smoking
• Successful in 20 of smokers
• Negligible cost

What are the consequences of making a Type I
error? What are the consequences of making a Type
II error?
32
Type I II errors
Example 2

• Trial of new brochure to help people quit smoking
• Successful in 20 of smokers
• Negligible cost

• Conclusion
• Can relax Type I error (say 0.10)
• Requires stringent Type II error (say 0.05)

33
Scenario 2
Three bits of information required to determine
the sample size
Type I II errors
Variation
Clinical effect
34
Clinical effect
Your Alternative hypothesis states that you
expect one group to have a different mean or
proportion to the other group, but how much by?

• From the literature
• From a pilot study
• Clinically judgement

• ? 15 change
• Change of ? 1 SD
• Interim analysis

35
Scenario 2
Three bits of information required to determine
the sample size
Type I II errors
Variation
Clinical effect
36
Variation
Is there a difference between the two means?
Mean 1
Mean 2
Systolic Blood Pressure
37
Variation
It depends upon the range of the distributions
Systolic Blood Pressure
38
Variation
To judge whether the difference between two means
is large or small, we compare it with some
measure of the variability of the distributions

39
Variation
Variability
All statistical tests are based on the following
ratio
Difference between parameters Test Statistic
v / ?n
As n ? v/?n ? Test statistic ?
40
Variation
v x Test
statistic n
Difference
2
41
Variation

• The test-statistic is usually
• Chi-squared for comparing two proportions
• Students t for comparing two means
• F-statistic for comparing two variances
• Z-statistic for comparing two correlation
  coefficients
• but may be more complicated
  

42
Scenario 2
Example for two means We wish to undertake an RCT
of an intervention to improve quality of life. At
the end of the study, the mean PCS of the SF-36
for the control group is expected to be 35. We
expect that in the intervention group, the mean
PCS will be 45. The standard deviation of the PCS
is 10.
43
(No Transcript)
44
1 Type I Error
1 Type II Error
45
Scenario 2
Example for two proportions In a prospective
study of hip protectors, we expect that in the
untreated group 10 of elderly people will suffer
a hip fracture. In the treated group we expect
this to reduce to 5.
46
(No Transcript)
47
Winepiscope
Winepiscope is available free of charge from
http//www.clive.ed.ac.uk/winepiscope/

48
Allowing for dropouts 
Nearly all studies have at least some subjects
who withdraw, are lost to follow up, or who die
If n is the sample size computed by the program,
and we expect lose d of subjects, then the
requires sample size is N is given by
N (100 x n) / (100 d)
49
Allowing for dropouts 
Example The sample size program tells us that we
need 120 in each group and we are expecting a 15
drop out.
N (100 x 120) / (100 15)
141
50
Is bigger better?  
For both descriptive and hypothesis testing
studies, the answer is yes.

1. Increasing the sample size will have no effect on
   Type I errors which are largely due to bias
   and/or confounding.

1. There is no point in having a larger sample size
   than that required for precision or power.

51
Is bigger better?  
For both descriptive and hypothesis testing
situations, the answer is yes. However

1. Increasing the sample size will have no effect on
   Type I errors which are largely due to bias
   and/or confounding.

1. There is no point in having a larger sample size
   than that required for precision or power.

52
For copies of this presentation
Please email Kylie Thomas at kylie.thomas_at_flinde
rs.edu.au