Information Mastery for the 21st Century

1
Information Mastery for the 21st Century
or
how I learned to stop worrying and love evidence
based medicine

• Peter C. Smith, MD
• Assistant Professor of Family Medicine, UCDHSC
• Director, BIGHORN Research Network
• Assistant Editor, Journal of Family Practice
• Assistant Editor, Family Physicians Inquiries
  Network
• Faculty, Rose Family Medicine Residency

2
WARNING!!!!

• THIS TALK CONTAINS.
• IDEAS!
• CONTROVERSIAL (MANY)
• DEBATABLE (ALL)
• NEW (SOME)
• QUOTATIONS
• RECYCLED (MOST)
• FROM OLD DEAD WHITE GUYS (MOST)
• MOVING TEXT
• DISTRACTING
• ANNOYING

3
Road Map

• Evidence Based Medicine (EBM)
• What It Is
• What It Is Not
• A Historical Perspective
• Why EBM?
• Now What?

4
This is going to be GREAT!

• There's nothing more exciting than science. You
  get all the fun of sitting still, being quiet,
  writing down numbers, paying attention. Science
  has it all.
• - Principle Seymour Skinner,
• The Simpsons

5
What Evidence Based Medicine IS

• A way of practicing medicine.
• The conscientious, explicit and judicious use of
  current best evidence in making decisions about
  the care of individual patients. --Sackett et
  al. JAMA 1993

6
What Evidence Based Medicine IS

• 1. Regularly asking specific clinical questions
  about patients problems
• 2. Efficiently searching existing sources of
  information for the answers
• 3. Appraising the quality validity of the
  answers
• 4. Implementing useful findings into everyday
  practice

7
What Evidence Based Medicine IS NOT

• Cookbook medicine
• Tyranny of Randomized Controlled Trials
• Telling doctors how to practice
• A substitute for sound reasoning, judgement, and
  knowledge of the patient
• A conspiracy to cut costs while depriving people
  of effective medical care
• Impossible

8
A Quick Timeline

• 17th Century EBM
• 18th Century EBM
• 19th Century EBM
• 20th Century EBM
• 21st Century EBM

9
17th Century EBM

• Sir William Petty 1623-1687
• (a) Father of Modern Economics and Political
  Statistics
• The results of care are what matter
• People with access to doctors fared no better
  than people without doctors.
• Not very popular!

10
18th Century EBM

• Pierre Alexander-Charles-Louise 1787-1872
• Applied numerical tools to clinical medicine to
  prove efficacy of treatment
• Observation comparison
• First to declare therapeutic phlebotomy as
  useless and likely harmful
• Radical Empiricism and Therapeutic Skepticism

11
19th Century EBM (part I)

• Ignatz Semmelweis (1818-1865)
• Midwives maternity ward mortality 1
• Doctors maternity ward mortality 34
• Careful observation and early application of
  statistical techniques indicated the cause
• Dirty hands (pre-germ theory)
• Disgraced for his ridiculous ideas
• 140 years later

12
19th Century EBM (part II)

• Sir James MacKenzie 1853-1925
• State of the Art 1880
• Standard of Care for Heart Murmurs and PVCs Bed
  rest
• Noticed that many murmurs and extra systoles
  were benign in general practice.
• millions saved from life in bed
• (and DVTs, bed sores, poverty?)

13
The Great Quantitative Debate

• Competing European Paradigms Imported to the
  United States
• The French/English Empiricists
• Observation and Comparison
• Focus on Patient Outcomes (POEMs)
• The German Etiologists (Koch)
• Laboratory Experimentation
• Focus on Disease and Etiology (DOE)

14
The Etiologists Flex their Muscles

• Abraham Flexner
• Charged with up-grading US higher education, then
  medical education
• 1910, The Flexner Report
• Brother Simon Flexner was
• A physician
• A Kochian etiologist

15
20th Century EBM

• Medical Science as we commonly know it
• Explosion in medical knowledge
• Largely driven by intermediate endpoints
• Blood sugar, BP, fetal heart rate, excision
• Influenced by Kochian perspective
• Not working
• Never has a nation spent so much to accomplish
  so little for so few. --Larry Green
• Late 20th Century resurgence of empirical
  approach
• Called, Evidence-Based Medicine

16
Road Map

• What EBM is
• What EBM is not
• A historical timeline - 21st Century EBM?
• Why EBM?

17
Why Practice EBM?

• An evidence-based approach... liberates you from
  a reliance on dogma and tradition, and it allows
  you to critically evaluate both traditional and
  alternative or complementary therapies in an
  even-handed manner. -Mark Ebell, 1999
• it is permissible to make a judgment after you
  have examined the evidence. In some circles it is
  even encouraged - Carl Sagan

18
Why EBM? Practice Variation

• Prostactectomy rates (per 100,000)
• Rhode Island 20
• Alaska 429
• Ratio 21 to 1!
• Are Rhode Islanders being negleted?
• Are Alaskans getting butchered?
• What the Hell is going on?!?!?!?!?

19
Why EBM? Practice Variation
Wennberg J, Center for the Evaluative Clinical
Sciences, Dartmouth Medical School. "Geography
and the Debate Over Medicare Reform," Web
Exclusive for Health Affairs, 2/02
20
Why EBM? Practice Variation

• Therapeutic decisions are based on
• Strong evidence from clinical trials
• 4
• Minimal evidence from studies but strong clinical
  consensus
• 45
• Neither evidence nor consensus but on personal
  opinion
• 51

Field MJ, Lohr KN. Guidelines for clinical
practice. Institute of Medicine. Washington, DC
National Academy Press 1992. p. 34-9.
21
Why EBM ? Different Frames of Reference

• Selection Bias
• Efficacy vs. Effectiveness

22
Why EBM? Different Frames of Reference
23
Why EBM? Different Frames of Reference

• The university medical centersees biased
  samples of one tenth of one percent of the sick
  adults, from which students of the health
  professions must get an unrealistic concept of
  medicines task - Kerr White

24
Why EBM? Different Frames of Reference

• A personal anecdote
• How do you treat chronic fluid behind the ear
  drum in a patient with allergies?
• Ask your friendly neighborhood Otologist
• You must get a CT scan to rule out
  naso-pharyngeal carcinoma!
• (I did - negative for diagnosis of
  naso-pharyngeal carcinoma, but positive diagnosis
  of anxiety!)

25
Why EBM? Different Frames of Reference

• Efficacy vs. Effectiveness
• Efficacy
• The ability to produce effect in controlled
  environment (e.g., RCT)
• Physics Frictionless Universe Economics All
  other things being equal
• Effectiveness
• Whether that efficacy persists when introduced
  into the real world.
• Too often ignored in medical research
• Patients are sicker we dont take our pills
  differences in gender, race, and age its too
  hard too expensive side effects etc.
• Can it work vs. Should I use it?

26
Why EBM? The Information Tsunami

• So much research, so little time
• Even the newest textbooks are 5-10 years out of
  date
• 6,935 articles a year in primary care alone 19
  articles a day, 365 days a year..
• FOR THE REST OF YOUR LIFE!

27
Why EBM? The Information Tsunami

• From Evidence Based Practice, Paul Glasziou
• University of Queensland Oxford

28
Why EBM? The Information Tsunami
Your First EBM Equation
29
The Equation Validity

• Internal Validity
• Does study measure what it says it does?
• External Validity
• Can the results be generalized to a larger
  population?
• (efficacy vs. effectiveness)

30
The Equation Validity

• Worksheets and Tools
• http//www.med.ualberta.ca/ebm/
• Study designs and statistics are ways to avoid
  BIAS. Less bias closer to the truth
• Dont get scared by statistics
• "In science as in love, too much concentration on
  technique can often lead to impotence. -P. L.
  Berger

31
Why EBM? The Information Tsunami

• So much crappy research, so little time
• Education...has produced a vast population able
  to read but unable to distinguish what is worth
  reading. --George Macaulay Trevelyan

32
Study Designs Acute MI treatment
60
N43
50
Studies with Significant Case-Fatality Rate Diff
erences
40
30
N45
20
10
N57
0
Blinded Randomization
Unblinded Randomization
Non-Random
Chalmers,et al. N Engl J Med 1983 3091358-61
33
The Equation Relevance

• Does the study...
• Address the issue in which youre interested?
• Provide...
• DOE (disease oriented evidence) like FEV1, BP,
  LAD patency, etc?
• or PROSE (Prescriptive Recommendations based on
  Substandard Evidence), i.e. an uncritical,
  non-systematic review?
• or POEM (patient-oriented evidence that matters,
  like mm, symptom reduction,quality of life)?

34
DOE vs. POEMs
35

• Effect on Patient-Oriented Outcomes
• Symptoms
• Functioning
• Quality of Life
• Lifespan

Valid Patient-Oriented Evidence

• Effect on Disease Markers
• A1c in diabetes
• MICs in infection
• BMD in osteoporosis

Disease-Oriented Evidence
Relevance of Outcome

• Effect on Risk Factors for Disease
• Improvement in markers (blood pressure,
  cholesterol)

• Highly Controlled Research
• Randomized Controlled Trials
• Systematic Reviews

• Physiologic Research
• Preliminary Clinical Research
• Case reports
• Observational studies

Uncontrolled Observations Conjecture
Validity of Evidence
Graphic from Allen Shaughnessy, PharmD EBM Is
it enough?
36
Levels of Evidence

• Centre for Evidence Based Medicine, Oxford
• Levels of evidence 1a,b,c2a,b,c3a,b45
• Complicated, confusing
• Strength of Recommendation Taxonomy
• S.O.R.T Useful, elegant
• Levels 1, 2, and 3

37
(No Transcript)
38
Strength of Recommendation Taxonomy (SORT)

• Effect on Patient-Oriented Outcomes
• Symptoms
• Functioning
• Quality of Life
• Lifespan

SORT A
SORT B

• Effect on Disease Markers
• A1c in diabetes
• MICs in infection
• BMD in osteoporosis

SORT C
Relevance of Outcome

• Effect on Risk Factors for Disease
• Improvement in markers (blood pressure,
  cholesterol)

• Highly Controlled Research
• Randomized Controlled Trials
• Systematic Reviews

• Physiologic Research
• Preliminary Clinical Research
• Case reports
• Observational studies

Uncontrolled Observations Conjecture
Validity of Evidence
Graphic from Allen Shaughnessy, PharmD EBM Is
it enough?
39
A Tale of 2 Pyramids
40
Why EBM? The Information Tsunami

• So much crappy research, so little time
• POEM Patient Oriented Evidence that Matters
• Topics that measure outcomes that are meaningful
  to patients (e.g. morbidity, mortality, quality
  of life) that could change practice if true
• Ebell M. et al. Finding POEMs in the Medical
  Literature. J Fam Pract 1999
• Reviewed 8,085 articles in 85 medical journals
  over 6 months
• Only 2.6 qualified as POEMs
• Still 211 articles 1.17 articles per day.
• FOR THE REST OF YOUR LIFE!

41
Why EBM? The Information Tsunami
Your First EBM Equation
42
Why EBM? The Information Tsunami

• Hunters and gatherers
• Information Gatherers
• Passively graze whatever comes along
• Journal subscriptions
• Email Alerts
• CME programs
• Pharmaceutical detailing
• Information Hunters
• Go out and get answers to questions

43
Just in Time learningInterns information
needs

• Setting 64 residents at 2 New Haven hospitals
• Method Interviewed after 401 consultations
• Questions
• Asked 280 questions (2 per 3 patients)
• Pursued an answer for 80 questions (29)
• Not pursued because
• Lack of time
• Forgot the question
• Sources of answers
• Textbooks (31), articles (21), consultants (17)

Green, Am J Med 2000
44
Fertile Fields Happy Hunting Grounds

• Information Masters do Hunting AND gathering
• The next best thing to knowing something is
  knowing where to find it. -Samuel Johnson
• Primary research (Medline, PubMed,Journals)
• Secondary sources of evidence
• EBM Books (Best Evidence)
• Journals (J. of Family Practice, ACP Journal
  Club)
• Electronic (FPIN/PEPID, Cochrane, DARE,
  InfoRetreiver, Bandolier, Medline EBM filters,
  etc.)

45
(Relevance x Validity) / Work YODA?

• I not only use all the brains that I have, but
  all that I can borrow. -- Woodrow Wilson
• Your Own Data Analyzer

46
Anti-YODAs

• PROSE (Proscriptive Recommendations based On
  Substandard Evidence)
• American Family Physician (AFP)
• Clinics of North America
• Post Graduate Medicine
• Old English J. of Esoterica (NEJM)
• throw away journals
• Up to Date

47
YODAs

• Cochrane Database
• Systematic reviews and meta-analysis
• Journal of Family Practice (JFP)
• J. of the American Board of FP (JABFP)
• British Medical Journal (BMJ)
• Best Evidence
• Evidence Based Practice Newsletter
• Bandolier
• InfoRetriever / InfoPointer
• Family Practice Information Network (FPIN)

48
The risk of PROSE

• Expertise is inversely related to the truth!
• Shaughnessy Slawson, BMJ 2002
• Looked at rates at which non-systematic review
  articles on diabetes mentioned the best evidence
  from the United Kingdom prospective diabetes
  study (UKPDS)

Joyce J. et alJAMA. 1998
49
PROSE vs POEMs
50
(No Transcript)
51
Why EBM? Costs

• Skyrocketing costs require us to eliminate waste
• A billion here, a billion there, and pretty soon
  you're talking about real money - Everett
  Dirksen
• My problem lies in reconciling my gross habits
  with my net income - Errol Flynn

52
Scariest Picture of the Year
DeVoe JE, Dodoo MS, Phillips RL, Green LA. Who
will have health insurance in the year 2025? Am
Fam Physicain 2005721989.
53
The Equation Work

• There comes a time in the affairs of man when he
  must take the bull by the tail and
• ...face the situation.
• -- W. C. Fields

54
Dimensions of Evidence
55
Family Physicians Inquiries Network (FPIN)

• A national, not-for-profit consortium of academic
  family physicians, residency programs and
  departments, medical librarians, and
  informaticians dedicated to answering 80 of
  primary care doctors questions with the best
  available evidence in 60 seconds or less
• WWW.FPIN.ORG

56
FPIN Clinical Inquiries

• Clinical Inquiries provide the ideal answers to
  clinical questions using a structured search,
  critical appraisal, authoritative
  recommendations, clinical perspective, and
  rigorous peer review, Clinical Inquiries deliver
  best evidence for point of care use.

57
A Case

• A 62 year old woman presents for an annual exam.
  Both her mother and her sister died from breast
  cancer and she is quite anxious about her risk
  for developing breast cancer. She has read about
  tamoxifen and wants to know if her physician will
  prescribe it for her.
• Clinical Question Does tamoxifen prevent
  breast cancer?
  

58
Clinical Inquiries from the Family Physicians
Inquiries Network
Does tamoxifen prevent breast cancer?Go to the
full document. Tamoxifen prevents breast cancer
in women older than 60 years and in younger women
with equally high risk because of breast disease
and reproductive and family history, but there is
no current evidence for or against long-term
survival or overall health benefits. (Grade A
Evidence) A 49 reduction in 5-year incidence
of invasive and noninvasive breast cancer but
increased risk for endometrial cancer, pulmonary
emboli, deep vein thrombosis, and cataracts. The
long-term benefits and overall health effects of
tamoxifen for primary prevention of breast cancer
remain unclear the ongoing International Breast
Cancer Intervention Study trial is designed to
address this question. Table Events in 5 years
in 1000 women with intact uteri Document Type
Evidence SummaryFrom Family Physicians
Inquiries Network Clinical InquiriesCitation
Meriwether RA. J Fam Pract 2001 Dec50(12)1023
PubMed
59
FPIN

• For Hunters
• PEPID
• Multi-use medical informatics product
• Medical knowledge
• Drug database
• Clinical decision support
• Clinical Inquiries integrated by hyperlink

• For Gatherers
• Clinical Inquiries
• In the Journal of Family Practice
• In the American Family Physician
• Evidence Based Practice Newsletter
• EBM reviews
• POEMs

60
FPIN
61
InfoPoems

• For Hunting InfoRetriever
• A database of filtered, synopsized evidence with
  an integrated search engine.
• JFP POEMs
• 2,828 Abstracts of Cochrane Systematic Reviews
• 2,393 Evidence-based Practice Guidelines
• 734 Decision Support Calculators
• 229 Diagnostic test calculators (unique
  combinations of
• 1,373 HPE calculators (unique combinations of
  symptom - disease - test)

• For Gathering
• DailyInfoPoems
• Regular email delivery of predigested research
  studies
• Only Patient Oriented Evidence that Matters
• Reviewed and critiqued by EBM experts
• www.infopoems.com

62
Why EBM?Lies my teacher taught me in medical
school

• Dos
• Routine episiotomy
• remember 50 of Lymph nodescancer
• Vaginal breech
• Patch corneal abrasions
• Abx for green nasal D/C
• Flecainide for ectopy
• Lidocaine for post MI prophylaxis
• HRT

• Donts
• VBACs
• Let babies sleep on their backs
• Use B-blockers in CHF
• Use steroids in prematurity
• Use MDIs for asthma exacerbations
• Treat pneumonia as an outpatient

63
Backlash!
64
Some competing ways of knowing

• ABM
• Authority Based Medicine
• Religions, philosophers, experts, friends,
  advertising
• RBM
• Rhetorically Based Medicine
• Based on the most eloquent advocate of a
  particular theory of disease (Miasma, Humors,
  Possession, Magnets, Imbalance, etc)
• MBM
• Microscopically Based Medicine
• Identification of etiologic agents by microscopy,
  treatment by their elimination (Koch, Pasteur)
• IBM
• Intuition Based Medicine
• Feels good, seems right, who I am as a person

65
21st Century EBM?

• The certainties of our age are the problems of
  the next. - R.H.Tawney, 1926
• Who What When Where How?

66
21st Century EBM

• Who gets left out?
• Those not included in RCTs
• Women, children, minorities, chronically ill with
  multiple co-morbidities, very elderly.

67
Who gets left out? DEBATE

• 400 vascular patients 80 y/o
• Randomized to optimal EBM care vs usual care
• 3 year f/u
• Intervention group
• More evidence-based Rx
• Lower TC, LDL, SBP, DBP
• No difference in CV events or deaths

68
21st Century EBM

• What gets studied and published?
• Medical research tends to focus on potentially
  profitable technological interventions
• At the expense of behavior change, counseling,
  relationships, changes in education and
  socio-economics, environmental change, public
  health, other soft modalities

69
Determinants of Health in the U.S.
McGinnis JM, Williams-Russo P, Knickman JR. The
case for more active policy attention to health
promotion. Health Affairs. 200221(2)78-93.
70
Allocation of Health Care Resourcesin the U. S.
McGinnis JM, Williams-Russo P, Knickman JR. The
case for more active policy attention to health
promotion. Health Affairs. 200221(2)78-93.
71
21st Century EBM

• When are all other things being equal?
• Where is the Frictionless universe?
• Clinical complexity, difficulty of practice
  change, efficacy vs. effectiveness

72
frictionless universe vs. real life

• Hypothetical 79 y/o female with COPD,
  osteoporosis, DM-II, HTN, OA
• 12 meds, 406/month
• Multiple potential drug interactions

73
frictionless universe vs. real life
74
And yet

• Evidence Based practise vs. usual care outcomes
  in stroke
• When cared for by E-B neurologists, patients were
  50 more likely to receive evidence-based
  interventions (Rx, stroke units, etc)
• And were 22 less likely to die in the next 90
  days.
  (Mitchell et al
  stroke 1996271937-43)

75
21st Century EBM

• When is research fraudulent?
• Ghost writing, selective publication, out-right
  fraud.

76
(No Transcript)
77
Whos to blame?
78
21st Century EBM

• How do we deal with conflicting evidence?

79
Road Map

• Evidence Based Medicine (EBM)
• What It Is
• What It Is Not
• A Historical Perspective
• Why EBM?
• Now What?

80
Some Wise Counsel by William Osler

• The greater the ignorance the greater the
  dogmatism.
• The good physician treats the disease the great
  physician treats the patient who has the
  disease.
• One of the first duties of the physician is to
  educate the masses not to take medicine.

81
A new Paradigm?

• EBM is NECESSARY but INSUFFICIENT
• Still need
• clinical judgment, interpersonal skills
• ability to elicit and integrate patient values
• professionalism
• 21st Century practice management skills
• AND something else, some new ways of knowing the
  right thing to do

82
Learning Collaboratives and sliding scales of
evidence

• For 45 years, the Cystic Fibrosis Foundation has
  kept track of the outcomes of every cystic
  fibrosis child cared for in the 117 cystic
  fibrosis centers around the U.S. By agreement
  with the centers, the data is kept confidential.
• In 2003, average life expectancy of people with
  cystic fibrosis was 33. In the top performing
  center it was 47.
• At the median CF center, the average FEV1 was 75
  of normal. At the top center it was 100 of
  normal.
• Qawande, A. The bell curve. New Yorker, Dec 2004

James W. Mold, STFM Plenary April 2007. Can
Family Medicine Become a Learning Community?
83
Audited Practices for Exemplary Practices

• Exemplar methods
• Very high expectations (e.g. normal FEV1)
• Patient involvement (e.g. anticipatory chest PT)
• Creative solutions to treatment challenges (e.g.
  electronic chest PT machine)
• Aggressive medical management

James W. Mold, STFM Plenary April 2007. Can
Family Medicine Become a Learning Community?
84
Sliding Scale of Evidence?
James W. Mold, STFM Plenary April 2007. Can
Family Medicine Become a Learning Community?
85
Web of Belief

• Willard Van Orman Quine

Sehon SR, Stanley DE. A phiosophical analysis of
the EBM debate. BMC Health Services. 2003314.
86
Web of Belief
Clinical Experience
Hypotheses
Adapted from James Klagge, Virginia Tech Dept. of
Philosophy, Web of Belief
87
Ancient Wisdom from the Buddha

• Do not believe in anything
• simply because you have heard it.
• simply because it is spoken and rumored by many.
• simply because it is found written in your books.
  
• merely on the authority of your teachers and
  elders.
• because it has been handed down for many
  generations.
• But after observation and analysis, when you find
  that anything agrees with reason and is conducive
  to the good and benefit of one and all, then
  accept it and live up to it.

88
Now What?

• Not everyone is meant to make a difference. But
  for me, the choice to lead an ordinary life is no
  longer an option.