Significance in Strategic Management Research

1
Significance in Strategic Management Research
Steve Werner Department of Management Bauer
College of Business University of Houston
2
Overview

• The problems of statistical significance.
• The solutions.
• Conclusions.

3
The Problems of Statistical Significance

• Statistical significance is not practical
  significance.
• The conventional cut-offs are arbitrary and
  misleading.
• Frequently occurring Type II errors.

4
Statistical Significance is not Practical
Significance

• Practical significance is effect size and
  importance.
• Related but not the same.
• Statistical significance largely affected by n.

5
Conventional Cut-offs are Arbitrary and Misleading

• Convention is
• plt.10 marginally significant
• plt.05 significant
• plt.01 highly significant
• plt.001very highly significant.
• plt.05 from Fisher (1925).
• Counters classical view of predetermining alpha.
• Simplifies hypothesis testing.

6
Frequently OccurringType II Errors

• Studies conclude no relationship because findings
  not significant.
• Power issues.
• This occurs in 60 of all studies (Hunter, 1997).

7
The Solutions

• Solutions to the practical significance problem.
• Solutions to the conventional cut-offs problem.
• Solutions to the frequently occurring Type II
  errors problem.

8
Solutions to the Practical Significance Problem

• Report confidence intervals.
• Report effect sizes.
• Avoiding using significance.
• Many journals still do not report effect sizes.
• SMJ 98.5 of studies report effect sizes.

9
Solutions to theArbitrary Cut-off Problem

• Go back to classical Neyman-Pearson hypothesis
  testing with researcher pre-determined alphas.
• Report actual p values (Fisher, 1956).
• Easier to accomplish now.
• SMJ 13.4 (35/262) of studies report some actual
  p values.

10
Solutions to theType II Error Problem

• Increase Power
• Conduct power analyses
• Increase sample size
• Increase effect size.
• Relax alpha.
• SMJ 65.6 (172/262) of studies use plt.10 as
  cut-off.
• Report actual p values.

11
(No Transcript)
12
Conclusion

• The problems of statistical significance.
• The solutions.
• Report actual p values.
• Consider p, power, effect size, sample size, and
  importance when evaluating research findings.