The Scientific Literature: Scientific Appraisal

1
The Scientific LiteratureScientific Appraisal

• John Bycroft
• Sarah Knight

2
Purpose of Scientific Literature

• Communication between researchers
• Historical record of progress of scientific
  research
• Avoids duplication of results and methods
• No Journal of Negative Results so some
  unsuccessful studies may be repeated.

3
Why Read the Scientific Literature?

• Keeping up with current scientific / clinical
  findings
• Evidence-based medicine decisions
• Developing new methodologies
• Find deficiencies in current knowledge
• Preparation of a literature review for a thesis
• Writing a review article
• Reviewing a paper for a journal

4
Why is Critical Appraisal Important?

• Determine the true results and conclusions
• Assess the validity of the results
• Assess the relative importance of the study

5
Types of Publication

• Peer reviewed journals
• Specialist (Urology, BJUI)
• Populist (Nature, Science etc)
• In-house publications
• Conference proceedings
• Peer reviewed
• Non-peer-reviewed
• Books
• Theses

6
Types of Article

• Original Article
• Review Article
• Technical Note
• Letters to Editor
• Short Communication
• Case Report

7
The Scientific Method

• Anecdote or Observation
• Literature Review
• Hypothesis
• Design of experiment under controlled conditions
  to test hypothesis
• Disproof or support
• Publication
• Hypothesis ? Theorem

8
Anatomy of a Paper

• Abstract
• Introduction/Background
• Methods Materials
• Patient Selection
• Results
• Discussion
• Conclusion

9
Abstract

• Summary of problem and main results
• Beware of assuming results from reading the
  abstract alone. This may not have been written by
  the authors and may be misleading or not contain
  all useful findings of the study
• You cant appraise a paper from the abstract alone

10
Introduction

• Understanding and presentation of previous work
  (by others and themselves)
• Clear statement of problem and hypothesis
• How they intend to answer problem or prove
  hypothesis

11
Methods Materials

• Detailed enough to allow repetition of study
• Experimental Design
• Statistical Tests
• A priori
• P value for null hypothesis
• Repeatability and reproducibility

12
Experimental Design

• Test and Controls
• Sample size
• power equation
• Patient Selection
• Inclusion/exclusion criteria
• Randomisation
• Matching (age/sex)
• Animal model
• Study Design
• Longitudinal/Retrospective
• Placebo controlled etc

13
Validation of Methods

• Specificity, precision, reliability
• Error of measurements
• Subjective or objective
• inter-, intra-rater variability
• Bias
• Diurnal or hormonal variations

14
Results

• Appropriate presentation
• Graphical
• Tabular
• Statistical Analyses
• Factual statements about observations

15
Discussion

• Brief summary of methods
• Explanation of limitations
• Implications of the results, individually and
  collectively
• Reference to previous studies
• Overall conclusion

16
Conclusions

• Supported by experimental results
• Objective
• Answer the hypothesis originally stated

17
Key Points for Scientific Appraisal

• What are you trying to achieve?
• Read through completely
• Find statement of problem
• Find key results and conclusions
• Determine strengths and weaknesses, e.g. major
  experimental design flaws
• Are conclusions supported by results?
• How are the conclusions or other findings
  relevant to you

18
Typical Journal Reviewers Checklist

• Comments on the novelty of these findings
• Do the findings of this study represent a major
  (rather than incremental) advance over previously
  published observations?
• Is this study hypothesis driven?

Courtesy of the Journal of Applied Physiology
19
Typical Journal Reviewers Checklist

• Is the rationale for this study logically
  presented?
• Are there major concerns regarding the
  experimental design and methods?
• Are the results clearly presented?
• Can the discussion be improved?