Book Reading: How to read a paper

1
Book Reading How to read a paper

• ??? ??? ??
• ??? ??? ??
• 91-07-03

2
How to read a paper

• The Medline database
• Getting your bearings (deciding what the paper is
  about)
• Assessing the methodological quality of published
  papers
• Statistics for the non-statistician. I Different
  types of data need different statistical tests
• Statistics for the non-statistician. II
  Significant relation and their pitfalls

3
How to read a paper

• Papers that report drug trails
• Papers that report diagnostic or screening tests
• Papers that tell you what things cost (economic
  analyses)
• Papers that summarize other papers (systematic
  review and meta-analysis)
• Papers that go beyond numbers (qualitative
  research)

4
The Medline database

• Over 10 million medical articles
• 1/3 are indexed in the Medline database
• How to trace articles
• Any word listed on the database
• Mesh (Medical subject heading)

5
The Medline database

• Other Databases
• AIDSLINE
• Allied and Alternative Medicine
• American Medical Association Journals
• ASSIA
• Cancer CD
• CDINAHL
• Cochrane Library
• Current Research in Britain

• DHData (formerly DHSS-Data)
• Embase
• HELMIS
• Psychlit
• Science Citation Index
• SHARE
• Toxline
• Unicorn

6
Getting your bearings

• The standard IMRAD format
• Introduction
• Why the authors decided to do this research
• Methods
• How they did it, and how they analysed their
  results
• Results
• What they found
• Discussion
• What the results mean

7
Getting your bearings

• Q1 Why was the study done, and what clinical
  question were the authors addressing?

8
Getting your bearings

• Q2 What type of study was done?
• Primary
• Experiments
• Clinical trials
• Surveys
• Secondary
• Overview
• Non-systemic summarize primary studies
• Systemic via a rigorous and predefined
  methodology
• Meta-analysis integrate the numerical data
• Guidelines
• Decision analysis
• Economic analysis

9
Getting your bearings

• Q3 Was this design appropriate to the research?

10
Getting your bearings

• Board fields or research
• Therapy
• Randomized controlled trial
• Diagnosis
• Cross sectional survey about new test and gold
  standard
• Screening
• Cross sectional survey
• Prognosis
• Longitudinal cohort study
• Causation
• Cohort or case-control study (depend on how rare
  the disease is)
• Case report

11
Getting your bearings

• Randomized Controlled Trials
• Is this drug better than placebo or a different
  drug for a particular disease?
• Is a leaflet better than verbal advice in helping
  patients make informed choices about the
  treatment options for a particular condition?

12
Getting your bearings

• Cohort Studies
• Two (or more) groups of people are selected on
  the basis of differences in their exposure to a
  particular agent, and followed up to see how many
  a each group develop a particular disease or
  other outcome.
• Does high blood pressure get better over time?
• What happens to infants who have been born very
  prematurely, in terms of subsequent physical
  development and educational achievement?

13
Getting your bearings

• Case-control studies
• Patients with a particular disease or condition
  are identified and matched with controls. Data
  are then collected on past exposure to a possible
  causal agent for the disease.
• Does the prone sleeping position increase the
  risk of cot death (the sudden infant death
  syndrome)?
• Dose whooping cough vaccine cause brain damage?
• Do overhead power cables cause leukemia?

14
Getting your bearings

• Cross sectional surveys
• What is the normal height of a 3 year old
  child?
• What to psychiatric nurse believe about the value
  of electroconvulsive therapy in severe
  depression?
• Is it true that half of all cases of diabetes are
  undiagnosed?

15
Getting your bearings

• Case reports

16
Getting your bearings

• The hierarchy of evidence
• Systematic reviews and meta-analysis
• Randomized controlled trials with definitive
  results (CI that do not overlap the threshold
  clinically significant effect)
• Randomized controlled trials with non-definitive
  results
• Cohort studies
• Case-control studies
• Cross sectional surveys
• Case reports

17
Assessing the methodological quality of published
papers

• Was the study original?

18
Assessing the methodological quality of published
papers

• Whom is the study about?
• How were the subjects recruited?
• Who was included in the study?
• Who was excluded from the study?
• Were the subjects studied in real life
  circumstance?

19
Assessing the methodological quality of published
papers

• Was the design of the study sensible?
• Critical appraisal
• What specific intervention or other maneuver was
  being considered, and what was it being compared
  with?
• What outcome was measured, and how?
• The level of enzyme v.s. the efficacy

20
Assessing the methodological quality of published
papers

• Was systemic bias avoided or minimized?
• Systematic bias anything that erroneously
  influences the conclusions about groups and
  distorts comparisons.

21
Assessing the methodological quality of published
papers

• Was assessment blind?

22
Assessing the methodological quality of published
papers

• Were preliminary statistical questions dealt
  with?
• Sample size
• What level of difference between the two groups
  would constitute a clinically significant effect?
• The mean and the standard deviation of the
  principal outcome variable
• Duration of follow up
• Completeness of follow up
• The reasons why patients withdraw

23
Statistics for the non-statistician. I Different
types of data need different statistical tests

• Have and authors set the scene correctly?
• Have they determined whether their groups are
  comparable, and if necessary, adjusted for
  baseline differences?
• A table showed the differences between the 2
  groups.
• What sort of data have they got, and have they
  used appropriate statistical tests?
• Parametric or non-parametric tests?
• Normal distribution? Non-normal (screwed) data?
• Transforming data to achieve a normal
  distribution is not cheating!!
• Using tests based on the normal distribution to
  analyze non-normally distributed data is
  definitely cheating!!

24
Statistics for the non-statistician. I Different
types of data need different statistical tests

• Have and authors set the scene correctly?
• Are the data analyzed according to the original
  protocol?
• Terminate an intervention trial prematurely for
  ethical reasons
• Raking over your data for interesting results
  (retrospective subgroup analysis)

25
Statistics for the non-statistician. I Different
types of data need different statistical tests

• Paired data, tails, and outliers
• Were paired tests performed on paired data?
• Was a two tailed test performed whenever the
  effect of an intervention could conceivably be a
  negative one?
• Tail the extremes of the distribution
• Were outliers analyzed with both common sense
  and appropriate statistical adjustments?

26
Statistics for the non-statistician. II
Significant relation and their pitfalls

• Correlation, regression, and causation
• Correlation v.s. regression
• If two things are not correlated, it will be
  meaningless for attempt a regression.
• r value (Persons product-moment correlation
  coefficient)
• The data should be normally distributed.
• The two datasets should be independent.
• Only a single pair of measurements should be made
  no each subject
• Every r value should be accompanied by a P value.

27
Statistics for the non-statistician. II
Significant relation and their pitfalls

• Correlation, regression, and causation
• Even if the r value is appropriate for a set of
  data, it does not tell you whether the relation
  is causal.
• Regression a mathematic equation that allow one
  variable to be predicted from another.
• Have assumptions been made about the nature and
  direction of causality?

28
Statistics for the non-statistician. II
Significant relation and their pitfalls

• Probability and confidence
• Have P values been calculated and interpreted
  appropriately?
• P lt0.05 statistically significant
• Plt0.01 statistically highly significant
• Have confidence intervals been calculated, and do
  the authors conclusions reflect them?
• The larger the trial, the narrower the confidence
  interval.

29
Papers that report drug trials

• Evidence and marketing
• Making decisions about treatment
• Identify, for this patient, the ultimate
  objective treatment
• Select the most appropriate treatment, using all
  available evidence
• Specify the treatment target

30
Papers that report drug trials

• Surrogate end points
• A variable which is relatively easily measured
  and which predicts a rare or distant outcome of
  either a toxic stimulus or a therapeutic
  intervention but which is not itself a direct
  measure of either harm or clinical benefit.
• A change in the surrogate end point does not
  itself answer the essential preliminary
  questions.
• The surrogate end point may not closely reflect
  the treatment target
• Over reliance on a single surrogate end point as
  a measure of therapeutic success usually reflect
  a narrow clinical perspective.
• Surrogate end points are often developed in
  animal models of disease

31
Papers that report drug trials

• How to get evidence out of a drug rep
• The STEP acronym
• Safety
• Tolerability
• Efficiency
• Price

32
Papers that report diagnostic or screening tests

• Does the paper validate the test?
• Is this test potentially relevant to my practice?
• Has the test been compared with a true gold
  standard?
• Did this validation stud include an appropriate
  spectrum of subjects?
• Has workup bias been avoided?
• Has expectation bias been avioded?

33
Papers that report diagnostic or screening tests

• Does the paper validate the test?
• Was the test shown to be reproducible?
• What are the features of the test as derived from
  this validatino study?
• Were confidence intervals given?
• Has a sensible normal range been given?
• Has this test been placed in the continuous other
  potential tests in the diagnostic sequence?

34
Papers that tell you what things cost (economic
analyses)

• Measuring costs and benefits of health
  interventions
• Questions of ask about an economic analysis
• Is the analysis based on a study that answers a
  clearly defined clinical question about an
  economically important issues?
• Whose viewpoint are costs and benefits being
  considered from?
• Have the interventions being compared been sowed
  to be clinically effective?
• Are the interventions sensible and workable in
  the setting where they are likely to be applied?

35
Papers that tell you what things cost (economic
analyses)

• Questions of ask about an economic analysis
• Which method of analysis was used, and was this
  appropriate?
• How were costs and benefits measured?
• Were incremental, rather than absolute, benefit
  considered?
• Was the here and now given precedence over the
  distant future?
• Was a sensitivity analysis performed?
• Were bottom line aggregate scores overused?

36
Paper that summarize other papers (systematic
review and meta-analyses)

• Advantages of systematic reviews
• Explicit methods limit bias in identifying and
  selecting studies
• Conclusions are more reliable and accurate
  because methods used
• Large amounts of information can be assimilated
  quickly by healthcare providers, researchers, and
  policymakers.
• Delay between research discoveries and
  implementation of effective diagnostic and
  therapeutic strategies may be reduced

37
Paper that summarize other papers (systematic
review and meta-analyses)

• Advantages of systematic reviews
• Results of different studies can be formally
  compared to establish generalisability of
  findings and consistency (lack of heterogeneity)
  of results
• Reasons for heterogeneity (inconsistency in
  results across studies) can be identified and new
  hypotheses generated about particular subgroups
• Quantitative systematic reviews (meta-analyses)
  increase the precision of the overall result.

38
Paper that summarize other papers (systematic
review and meta-analyses)

• Evaluating systematic reviews
• Can you find an important clinical question which
  the review addressed.
• Was a through search done of the appropriate
  databases and were other potentially important
  sourced explored?
• Was methodological quality assessed and the
  trails weighted accordingly?
• How sensitive are the results to the way the
  review has been done?
• Have the numerical results been interpreted with
  common sense and due regard to the broader aspect
  of the problem?

39
Paper that summarize other papers (systematic
review and meta-analyses)

• Meta-analysis for the non-statiscian
• ?? 95CI
• ???line of no effect
• ????pooling result

40
Paper that summarize other papers (systematic
review and meta-analyses)

• Explaining heterogeneity
• Homogeneity

41
Paper that go beyond numbers (qualitative
research)

• What is qualitative research?
• A finding or a result is more likely to be
  accepted as a fact if it is quantified than if it
  is not.

42
Paper that go beyond numbers (qualitative
research)

• Evaluating papers that describe qualitative
  research?
• Did the paper describe an important clinical
  problem addressed via a clearly formulated
  question?
• Was a qualitative approach appropriate?
• How were the setting and the subjects selected?
• What was the researchers perspective, and has
  this been taken into account?
• What methods did the researcher use for
  collecting data and are these described in
  enough detail?

43
Paper that go beyond numbers (qualitative
research)

• Evaluating papers that describe qualitative
  research?
• What methods did the researcher use to analyze
  the data and what quality control measures were
  implemented?
• Are the results credible, and if so, are they
  clinically important?
• What conclusions were drawn, and are justified by
  the results?
• Are the findings of the study transferable other
  clinical settingd?