Statistical Power And Sample Size Calculations - PowerPoint PPT Presentation

1 / 58
About This Presentation
Title:

Statistical Power And Sample Size Calculations

Description:

When Do You Need Statistical Power Calculations, And Why? ... data described above was gathered using a 1m x 10m (10 m2) quadrat and the total ... – PowerPoint PPT presentation

Number of Views:2468
Avg rating:3.0/5.0
Slides: 59
Provided by: micha558
Category:

less

Transcript and Presenter's Notes

Title: Statistical Power And Sample Size Calculations


1
Statistical Power And Sample Size Calculations
Minitab calculations
Manual calculations
2
When Do You Need Statistical Power Calculations,
And Why?
  • A prospective power analysis is used before
    collecting data, to consider design sensitivity .

3
When Do You Need Statistical Power Calculations,
And Why?
  • A retrospective power analysis is used in order
    to know whether the studies you are interpreting
    were well enough designed.

4
What Is Statistical Power?Essential concepts
  • the null hypothesis Ho
  • significance level, a
  • Type I error
  • Type II error

5
Statistical Hypothesis Testing
  • When you perform a statistical hypothesis test,
    there are four possible outcomes

6
Statistical Hypothesis Testing
  • whether the null hypothesis (Ho) is true or false
  • whether you decide either to reject, or else to
    retain, provisional belief in Ho

7
Statistical Hypothesis Testing
8
When Ho Is True And You Reject It, You Make A
Type I Error
  • When there really is no effect, but the
    statistical test comes out significant by chance,
    you make a Type I error.
  • When Ho is true, the probability of making a Type
    I error is called alpha (a). This probability is
    the significance level associated with your
    statistical test.

9
When Ho is False And You Fail To Reject It, You
Make A Type II Error
  • When, in the population, there really is an
    effect, but your statistical test comes out
    non-significant, due to inadequate power and/or
    bad luck with sampling error, you make a Type II
    error.
  • When Ho is false, (so that there really is an
    effect there waiting to be found) the probability
    of making a Type II error is called beta (ß).

10
The Definition Of Statistical Power
  • Statistical power is the probability of not
    missing an effect, due to sampling error, when
    there really is an effect there to be found.
  • Power is the probability (prob  1 - ß) of
    correctly rejecting Ho when it really is false.

11
Calculating Statistical PowerDepends On
  • the sample size
  • the level of statistical significance required
  • the minimum size of effect that it is reasonable
    to expect.

12
How Do We Measure Effect Size?
  • Cohen's d
  • Defined as the difference between the means for
    the two groups, divided by an estimate of the
    standard deviation in the population.
  • Often we use the average of the standard
    deviations of the samples as a rough guide for
    the latter.

13
Cohen's Rules Of Thumb For Effect Size
14
Calculating Cohens d
14
15
Calculating Cohens d
15
16
Calculating Cohens d from a t test
16
17
Conventions And Decisions About Statistical Power
  • Acceptable risk of a Type II error is often set
    at 1 in 5, i.e., a probability of 0.2.
  • The conventionally uncontroversial value for
    adequate statistical power is therefore set at
    1 - 0.2  0.8.
  • People often regard the minimum acceptable
    statistical power for a proposed study as being
    an 80 chance of an effect that really exists
    showing up as a significant finding.

18
Next Week
  • Statistical Power Analysis In Minitab

19
Statistical Power Analysis In Minitab
  • Stat gt Power and Sample Size gt

20
Statistical Power Analysis In Minitab
Note that you might find web tools for other
models. The alternative normally involves
solving some very complex equations.
  • 1-Sample Z
  • 1-Sample t
  • 2-Sample t
  • 1 Proportion
  • 2 Proportions
  • One-Way ANOVA
  • 2-Level Factorial Design
  • Plackett-Burman Design
  • Recall that a comparison of proportions equates
    to analysing a 22 contingency table.

21
Statistical Power Analysis In Minitab
Note that you might find web tools for other
models. The alternative normally involves
solving some very complex equations.
Simple statistical correlation analysis online
See Test 28 in the Handbook of Parametric and
Nonparametric Statistical Procedures, Third
Edition by David J Sheskin
22
Factors That Influence Power
  • Sample Size
  • alpha  
  • the standard deviation

23
Using Minitab To Calculate Power And Minimum
Sample Size
  • Suppose we have two samples, each with n  13,
    and we propose to use the 0.05 significance level
  • Difference between means is 0.8 standard
    deviations (i.e., Cohen's d  0.8)
  • All key strokes in printed notes

24
Using Minitab To Calculate Power And Minimum
Sample Size
Note that all parameters, bar one are
required. Leave one field blank. This will be
estimated.
25
Using Minitab To Calculate Power And Minimum
Sample Size
  • Power and Sample Size
  • 2-Sample t Test
  • Testing mean 1  mean 2 (versus not )
  • Calculating power for mean 1  mean 2
    difference
  • Alpha  0.05 Assumed standard deviation  1
  • Sample
  • Difference Size Power
  • 0.8 13 0.499157
  • The sample size is for each group.

Power will be 0.4992
26
Using Minitab To Calculate Power And Minimum
Sample Size
  • If, in the population, there really is a
    difference of 0.8 between the members of the two
    categories that would be sampled in the two
    groups, then using sample sizes of 13 each will
    have a 49.92 chance of getting a result that
    will be significant at the 0.05 level.

27
Using Minitab To Calculate Power And Minimum
Sample Size
  • Suppose the difference between the means is 0.8
    standard deviations (i.e., Cohen's d  0.8)
  • Suppose that we require a power of 0.8 (the
    conventional value)
  • Suppose we intend doing a one-tailed test, with
    significance level 0.05.
  • All key strokes in printed notes

28
Using Minitab To Calculate Power And Minimum
Sample Size
Select Options to set a one-tailed test
29
Using Minitab To Calculate Power And Minimum
Sample Size
30
Using Minitab To Calculate Power And Minimum
Sample Size
  • Power and Sample Size
  • 2-Sample t Test
  • Testing mean 1  mean 2 (versus gt)
  • Calculating power for mean 1  mean 2
    difference
  • Alpha  0.05 Assumed standard deviation  1
  • Sample Target
  • Difference Size Power Actual Power
  • 0.8 21 0.8 0.816788
  • The sample size is for each group.

Target power of at least 0.8
31
Using Minitab To Calculate Power And Minimum
Sample Size
  • Power and Sample Size
  • 2-Sample t Test
  • Testing mean 1  mean 2 (versus gt)
  • Calculating power for mean 1  mean 2
    difference
  • Alpha  0.05 Assumed standard deviation  1
  • Sample Target
  • Difference Size Power Actual Power
  • 0.8 21 0.8 0.816788
  • The sample size is for each group.

At least 21 cases in each group
32
Using Minitab To Calculate Power And Minimum
Sample Size
  • Power and Sample Size
  • 2-Sample t Test
  • Testing mean 1  mean 2 (versus gt)
  • Calculating power for mean 1  mean 2
    difference
  • Alpha  0.05 Assumed standard deviation  1
  • Sample Target
  • Difference Size Power Actual Power
  • 0.8 21 0.8 0.816788
  • The sample size is for each group.

Actual power 0.8168
33
Using Minitab To Calculate Power And Minimum
Sample Size
Suppose you are about to undertake an
investigation to determine whether or not 4
treatments affect the yield of a product using 5
observations per treatment. You know that the
mean of the control group should be around 8, and
you would like to find significant differences of
4. Thus, the maximum difference you are
considering is 4 units. Previous research
suggests the population s is 1.64.
34
Using Minitab To Calculate Power And Minimum
Sample Size
35
Using Minitab To Calculate Power And Minimum
Sample Size
Power 0.83
Power and Sample Size One-way ANOVA Alpha 0.05
Assumed standard deviation 1.64 Number of
Levels 4 SS Sample
Maximum Means Size Power Difference 8
5 0.826860 4 The sample size is
for each level.
36
Using Minitab To Calculate Power And Minimum
Sample Size
To interpret the results, if you assign five
observations to each treatment level, you have a
power of 0.83 to detect a difference of 4 units
or more between the treatment means. Minitab
can also display the power curve of all possible
combinations of maximum difference in mean
detected and the power values for one-way ANOVA
with the 5 samples per treatment.
37
Next Week
Manual Calculations of Power
38
Sample Size Equations
  • Five different sample size equations are
    presented in the printed notes.
  • For obvious reasons, only one is explored in
    detail here.

39
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
  • Calculate an initial sample size using the
    following equation

recall
40
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
  • Calculate an initial sample size using the
    following equation

41
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
42
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
To obtain the adjusted sample size estimate,
consult the correction table in the printed
notes. n is the uncorrected sample size value
from the sample size equation. n is the
corrected sample size value. See the example
below.
43
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
Additional correction for sampling finite
populations. The above formula assumes that the
population is very large compared to the
proportion of the population that is sampled. If
you are sampling more than 5 of the whole
population then you should apply a correction to
the sample size estimate that incorporates the
finite population correction factor (FPC). This
will reduce the sample size.
44
Determining The Necessary Sample Size For
Estimating A Single Population Mean Or A Single
Population Total With A Specified Level Of
Precision.
45
Example
  • Objective Restore the population of species Y in
    population Z to a density of at least 30
  • Sampling objective Obtain estimates of the mean
    density and population size of 95 confidence
    intervals within 20 () of the estimated true
    value.
  • Results of pilot sampling 
  • Mean ( )   25
  • Standard deviation (s)  7

46
Example
  • Given The desired confidence level is 95 so
    the appropriate Za from the table above is 1.96.
    The desired confidence interval width is 20
    (0.20) of the estimated true value. Since the
    estimated true value is 25, the desired
    confidence interval (B) is 25 x 0.20  5.

47
Example
  • Calculate an unadjusted estimate of the sample
    size needed by using the sample size formula

Round 7.53 up to 8 for the unadjusted sample size.
48
Example
  • To adjust this preliminary estimate, go to the
    sample size correction table and find n  8 and
    the corresponding n value in the 95 confidence
    level portion of the table. For n  8, the
    corresponding value is n  15.

49
Example
50
Example
  • The corrected estimated sample size needed to be
    95 confident that the estimate of the population
    mean is within 20 (5) of the true mean is 15.

51
Example
  • Additional correction for sampling finite
    populations.
  • If the pilot data described above was gathered
    using a 1m x 10m (10 m2) quadrat and the total
    population being sampled was located within a 20m
    x 50m macroplot (1000 m2) then N  1000m2/10m2  1
    00.

52
Example
  • The corrected sample size would then be

The new, FPC-corrected, estimated sample size to
be 95 confident that the estimate of the
population mean is within 20 (5) of the true
mean is 13.
53
Text
  • Sample size calculations in clinical research
  • edited by Shein-Chung Chow, Jun Shao, Hansheng
    Wang
  • New York Marcel Dekker, 2003
  • Long loan Robinson Books Level 4
  • 610.72 SAM
  • see also the conventional notes

54
Bibliography 1 of 2
  • Also see Tutorial in Quantitative Methods for
    Psychology Volume 3, no 2 (2007) Special issue
    on statistical power.
  • Editors note The Uncorrupted Statistical Power
  • Statistical Power An Historical Introduction
  • Understanding Power and Rules of Thumb for
  • Determining Sample Sizes
  • A Short Tutorial of GPower

55
Bibliography 2 of 2
  • Computing the Power of a t Test
  • Also see
  • Non-central t Distribution and the Power of the t
    Test A Rejoinder
  • Understanding Statistical Power Using Non-central
    Probability Distributions Chi-squared,
    G-squared, and ANOVA
  • Power Estimation in Multivariate Analysis of
    Variance
  • A Power Primer

56
Caveat
  • It is well known that statistical power
    calculations can be valuable in planning an
    experiment. There is also a large literature
    advocating that power calculations be made
    whenever one performs a statistical test of a
    hypothesis and one obtains a statistically
    non-significant result.

57
Caveat
  • Advocates of such post-experiment power
    calculations claim the calculations should be
    used to aid in the interpretation of the
    experimental results. This approach, which
    appears in various forms, is fundamentally
    flawed.

58
Caveat
  • The paper documents that the problem is
    extensive and presents arguments to demonstrate
    the flaw in the logic.
  • The abuse of power The pervasive fallacy of
    power calculations for data analysis, Hoenig JM,
    Heisey DM American Statistician, 551, 19-24,
    2001
Write a Comment
User Comments (0)
About PowerShow.com