The Presentation of Statistics in Clinical and Health Psychology Research - PowerPoint PPT Presentation

Loading...

PPT – The Presentation of Statistics in Clinical and Health Psychology Research PowerPoint presentation | free to view - id: 779e66-N2Q3M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

The Presentation of Statistics in Clinical and Health Psychology Research

Description:

The Presentation of Statistics in Clinical and Health Psychology Research Jeremy Miles Department of Health Sciences Susanne Hempel Centre for Reviews and Dissemination – PowerPoint PPT presentation

Number of Views:61
Avg rating:3.0/5.0
Slides: 35
Provided by: Jeremy271
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: The Presentation of Statistics in Clinical and Health Psychology Research


1
The Presentation of Statistics in Clinical and
Health Psychology Research
  • Jeremy Miles
  • Department of Health Sciences

Susanne Hempel Centre for Reviews and
Dissemination
2
Introduction
  • Statistics in clinical and health psychology
  • Appropriate statistics used
  • Statistics appropriately presented
  • Graphical display
  • Verbal presentation

3
Methodology
  • Reviewed 2003 volumes (4 issues) of
  • British Journal of Clinical Psychology
  • British Journal of Health Psychology
  • Looking for
  • Errors of statistical presentation /
    interpretation
  • Potential areas of improvement

4
Results
  • BJCP 29 papers reviewed
  • BJHP 31 papers reviewed
  • 5 excluded (qualitative, narrative review)
  • Wide range of problems identified
  • Emerging themes
  • P-values
  • Inferential statistics
  • Effect Sizes
  • Reliability
  • Other Issues
  • 2 papers with no issues

5
Statistical Significance
6
Statistical Significance
  • Confusing and controversial issue
  • Misunderstood by students, researchers, teachers,
    textbook authors
  • (Broadly) two rival approaches to probability
  • Fisher report exact significance value
  • Neyman-Pearson lt0.05, or not
  • These are incompatible(!)
  • (Ignoring Bayes ignoring meanings of probability)

7
A Bastardised Approach
  • (From Gigerenzer, 1992)
  • The two approaches are misunderstood, and
    combined
  • We must report the exact p
  • We must present results as lt0.xx
  • Recommended
  • Exact probability values (e.g. Wilkinson, et al,
    1999)

8
Results of p-value reporting
  • BJCP 8 out of 29 reported exact p-values
  • 1 used strict N-P approach
  • BJHP 4 out of 26 reported exact p-values

9
More on P-Values
  • 2 papers reported p lt 0 (.00)
  • True values were 0.000040, 0.000007
  • Several reported arbitrary cutoffs
  • lt0.07, lt0.02
  • Incorrect, but not deceptive

10
Misleading?
  • Not using exact p-values sometimes appears fishy
  • Exact p-values for all except where p 0.049,
    reported as p lt 0.05
  • Gave p gt 0.05 (p 0.057), p lt 0.05 (p 0.048)
  • P lt 0.01 when p 1 10-19 (others in same paper
    reported as p lt 0.001)
  • p 0.0104, described as lt 0.01, p 0.0123
    described as lt0.05

11
Finally Mistakes
  • Good old errors
  • Very hard for readers and reviewers to spot, but
    still
  • F (1, 69) 4.58, p lt 0.001
  • No, p 0.035
  • F (1.76, 142.51) 3.026, p .058.
  • No, p 0.084
  • F 4.02, (df not given, but are 2, 62), p
    0.05. (information in table)
  • No, p 0.022

12
Inferential Statistics
13
Reporting Test Statistics
  • Most people cant interpret a test statistic
  • Even fewer are interested
  • Why report a test statistic exactly, and not the
    exact p?
  • no significant interaction of both variables,
    F (1,67) .289. No p-value given (its 0.59)
  • F without df
  • No use at all (unless df can be worked out, but
    can be tricky or ambiguous)

14
Standard Errors
  • Standard error is the standard deviation of the
    sampling distribution
  • Used to calculate t (and hence p-value) and CIs
  • 95 CIs given by
  • Value depends on df
  • df 5, ta/2 2.57
  • df 100, ta/2 1.98
  • Standard error has little use.

15
Graph shows mean /- 1 SE. SE Mean is not showing
anything useful
16
Graph shows mean /- standard error. Data are
repeated measures.
17
Confidence Intervals
  • Generally recommended that confidence intervals
    are reported
  • Better idea of the likely value in the population
  • Not significant ? no effect
  • Appropriate confidence intervals
  • BJCP 3 (of 29)
  • BJHP 4 (of 26)

18
Inappropriate Confidence Intervals / Standard
Errors
  • Compare two groups
  • Appropriate standard error / confidence interval
    is of the difference , not of each group

19
Independent groups study Significant difference?
Yes. t 2.7, df 18, p 0.016, difference
2.7, 95 CIs 0.60, 4.80
20
Repeated measures study Significant difference?
t 2.25, df 9, p 0.051 Difference 2.7, 95
CIs -0.02, 2.25
Trick question. Its the same graph, and I
havent given you enough information
21
Effect Sizes
22
Effect Sizes
  • More statistically significant larger, more
    important effect?
  • No
  • Effect sizes describe the size of the effect
  • r, d, h2, R2

Yes No
BJCP 4 16
BJHP 7 10
23
Reliability Reporting
24
Reliability Reporting
  • Small, but important
  • Reliability is not a property of a test
  • It is a property of a test, in a population, at a
    particular time
  • Reliability should always be evaluated, and
    presented

All Some None
BJCP 5 4 14
BJHP 6 3 11
25
Stepwise Regression
  • Almost never appropriate
  • Small differences in samples can lead to large
    differences in results
  • 1 paper discusses differences between two
    stepwise regressions
  • Df are wrong (hence F, and p are also wrong)
  • Use of stepwise regression
  • BJCP 1
  • BJHP 2 (one not described as stepwise)

26
A Collection of Smaller Issues
27
Distributional Assumptions
  • Very few tests assume normal distribution of the
    variables
  • When sample sizes are at least moderate, normal
    distribution unimportant
  • Kolmogorov-Smirnov test examines significant
    difference from normality
  • Not important difference from normality (Field?)
  • 2 papers (BJCP) used the KS test
  • Non-parametric tests

28
Other Miscellany
  • Mann-Whitney test described as comparing medians
    (it doesnt necessarily)
  • Principal components analysis described as
    exploratory factor analysis (its not)
  • Expected values of chi-square test violated
  • Arithmetical errors in chi-square test
  • Correlation used as measure of agreement
  • We all know that it isnt
  • Inappropriate dichotomisation of continuous
    variables
  • Never necessary

29
Hall of Shame
30
Conclusions
31
Summary
  • Picture isnt rosy
  • Errors are not limited to psychology
  • Garcia-Berthou and Alcaraz (2004) found errors in
    Nature and the British Medical Journal
  • There are a lot of areas for improvement

32
Solutions? Short Term
  • More statistical refereeing?
  • More guidelines for reviewers
  • More reviewers with expertise in statistics
  • BJCP and BJEP have statistical reviewers
  • Rapid response?
  • Could be set up with the electronic journals
  • Work in other fields

33
Solutions? Long Term
  • Statistical / methodological training?
  • Undergraduate? Postgraduate? CPD?
  • Work more closely with statisticians?
  • Common in other fields MSc in Medical
    Statistics is possible, MSc in Psychological
    Statistics is not

34
Final Thought
  • Aaagggghhhhh!
  • We just did a piece of qualitative research?
About PowerShow.com