Making Sense

1
Making Sense Of Statistical Significance Lesson
10.3
2
Choosing a level of significance The purpose of a
test of significance is to give a clear statement
of the degree of evidence provided by the sample
against the null hypothesis. The P-value does
this. Choose a by asking how much evidence is
required to reject H0. This depends on two
circumstances How plausible is H0? What are the
consequences of rejecting H0?
3
There is no sharp border between significant
and insignificant only increasingly strong
evidence as the P-values decrease. Beware of
placing too much weight on traditional values of
a.
4
Statistical inference is not valid for all sets
of data. Always ask how the data were produced.
And dont be too impressed by P-values on a
printout until you are confident that the data
deserve a formal analysis.
5
Once you have a hypothesis, design a study to
search specifically for the effect you now think
is there. If the result of this study is
statistically significant, you have real evidence.
6
Very small effects can be highly significant
(small P), especially when a test is based on a
large sample. A statistically significant effect
need not be practically important. Plot the data
to display the effect you are seeking, and use
confidence intervals to estimate the actual value
of parameters.