Power Analysis

1
Power Analysis
Lecture (1-2) theory and examples 4th floor lab
(2-3) group work on analysis Stats lab (3-5)
computer analysis of duckweed data
2
An example Fox hunting in the UK
3

• Does fox hunting reduce fox populations?
• Hunt banned (one year only) in 2001 because of
  foot-and-mouth disease.
• Baker et al. (2002) examined whether the fox
  population increased in areas where it used to be
  hunted. No effect (p0.474, alpha0.05, n157).
• Aebischer et al. questioned whether the sample
  size was high enough to detect an effect.

Baker et al. 2002. Nature 419 34 Aebischer et
al. 2003. Nature 423 400
4
Power analysis Quantifies the likelihood of
detecting a REAL effect with the given
experimental design (including sample size).
5
Ho Treatments A and B the same HA Treatments
A and B different
6
Critical value at alpha0.05
Points on this side, only 5 chance from
distribution A.
Frequency
A
Area 5
A could be control treatment B could be
manipulated treatment
7
If null hypothesis true, A and B are identical
Probability that any value of B is significantly
different than mean of A 5 type 1 error
A
B
Probability that any value of B will be not
significantly different from mean of A 95
8
What you say
Decide NOT significantly different (do not reject Ho) Decide significantly different (reject Ho)
Ho true (same) Type 1 error
Ho false (different) Type 2 error
Reality
9
If null hypothesis false, two distributions are
different
Probability that any value of B is significantly
different than A 1- beta power
A
B
Probability that any value of B will be not
significantly different from A beta
likelihood of type 2 error
10
Effect size
A
B
Effect size Difference between means expressed
in terms of SD. e.g. Effect size 2 tells us
that the means are 2 SD units apart.
11
1. Power increases as effect size increases
Power
Effect size
A
B
Beta likelihood of type 2 error
12
2. Power increases as alpha increases
Power
A
B
Beta likelihood of type 2 error
13
3. Power increases as sample size increases
Low n
A
B
14
3. Power increases as sample size increases
High n
A
B
15
Alpha
Effect size
Power
Sample size
How much power is enough? Rule of thumb gt 80
16
Group exercise using p238 Krebs 1. What is the
power of the following t-test? (two-tailed at
alpha0.05) mean n SD A 12 7 1.0 B 10 7 1.
0 2. Why do ecologists try to have at least n3
in experiments?
17
Types of power analysis A priori Useful for
setting up a large experiment with some pilot
data Posteriori Useful for deciding how
powerful your conclusion is (definitely? Or
possibly). In manuscript writing, peer reviews,
etc.
18
Example 1 Wildlife monitoring programs (a priori)
www.mp2-pwrc.usgs.gov/powcase
19
To detect a 3 decline with 90 power

• How many sites should you monitor?
• How intensively should you monitor sites?
• How many years should you monitor sites?

20
Number of sites needed to detect 3 decline with
90 power
Intensity
Number of sites needed
21
Number of sites needed to detect 3 decline with
90 power
Intensity
Number of sites needed
Is it more cost effective to monitor many sites
at the lowest intensity (1) or a few sites at the
highest intensity (5)? Always?
22
Example 2 Fox hunting in the UK (posteriori)
23
157 plots where the fox population monitored.
If hunting had no effect, expect 50 of plots to
show increases, 50 to show decreases. If
hunting had an effect, expect 63 of plots to
show increase (as 63 of plots in hunted
areas). Effect size if hunting affected fox
populations 13 (63-50)
24
157 plots where the fox population monitored.
If hunting had no effect, expect 50 of plots to
show increases, 50 to show decreases. If
hunting had an effect, expect 63 of plots to
show increase (as 63 of plots in hunted
areas). Effect size if hunting affected fox
populations 13 (63-50) Power 0.95 !
25
Is there such a thing as too much power?