How do CME speakers use research results to support therapeutic recommendations A quantitative and q

1
How do CME speakers use research results to
support therapeutic recommendations?A
quantitative and qualitative study
SACME Rancho Mirage, CA April, 2009

• Michael Allen, Tanya Hill, Richard
  Handfield-Jones, Mike Fleming, Doug Sinclair, Tom
  Elmslie

2
Research team

• Dalhousie CME
• Michael Allen
• Tanya Hill
• Michael Fleming
• Doug Sinclair
• University of Ottawa CME
• Richard Handfield-Jones
• Foundation for Medical Practice Education
• Tom Elmslie

3
Research Team
Im Doug Sinclair from Dalhousie CME. I can find
time to do anything.
Im Richard Handfield-Jones, the PI from
University of Ottawa CME. Im in the army now and
AWOL.
Im Mike Fleming from Dalhousie CME and Im not
sure how I got involved in this.
Im Tanya Hill the research associate from
Dalhousie CME. I dont care who says theyre the
PI, Im running this show.
Im Mike Allen, the PI from Dalhousie CME
Im Tom Elmslie. Im an ideas man and helped with
experimental design.
4
Funding

• Royal College of Physicians and Surgeons
• of Canada
• Nova Scotia Department of Health through
• Drug Evaluation Alliance of Nova Scotia

5
Outline

• Background
• Objectives
• Methods
• Results
• Key messages

6
Think about this RCT

• Outcomes non-fatal myocardial infarction and
  death from coronary heart disease
• Subjects No history of coronary heart disease
  (primary prevention)
• Duration 3.3 years
• N 5100 patients in the control and 5100 patients
  in the drug group
• The patient characteristics are
• 80 male
• mean age 63 yrs.
• mean blood pressure 164/95

7
What is your interpretation of the following
results? How likely might you be to prescribe or
take the drug based on each one?

• The drug led to a 36 decrease in the incidence
  of non-fatal MI and CHD death (relative risk
  reduction).
• The drug decreased the rate of non-fatal MI and
  CHD death from 3.0 to 1.9, an absolute risk
  reduction of 1.1.
• You would have to treat 94 patients for 3.3 years
  to prevent one non-fatal MI or a death from CHD
  (number needed to treat).
• The 95 confidence intervals around the previous
  result (ie, treat 94 patients for 3.3 years to
  avoid one non-fatal MI or CHD death) are 60 and
  215.
• At the end of 3.3 years, 97.0 of patients who
  don't take the drug will remain free of a cardiac
  event and 98.1 of patients who take the drug
  will remain free of a cardiac event (inverse
  absolute RR).

8
Patients not having MI or dying Placebo
9
Patients not having MI or dying Treatment
10

• Forrow L, Taylor WC, Arnold RM. Absolutely
  relative How research results are summarized can
  affect treatment decisions. Am J Med 1992
  92121-4.
• Naylor CD, Chen E, Strauss B. Measured
  enthusiasm Does the method of reporting trial
  results alter perceptions of therapeutic
  effectiveness? Ann Intern Med 1992 117916-21.
• Nikolajevic-Sarunac J, et al. Effects of
  information framing on the intentions of family
  physicians to prescribe long-term hormone
  replacement therapy. J Gen Intern Med 1999
  14(10)591-598
• Nexoe J, Kristiansen IS, Gyrd-Hansen D, Nielsen
  JB. Influence of number needed to treat, costs
  and outcome on preferences for a preventive drug.
  Fam Pract 2005 22(1)126-131.

11
Objectives

• To determine
• How much emphasis speakers place on presenting
  data in absolute vs relative terms
• How familiar FPs and speakers are with
  statistical terms (especially absolute and
  relative terms)
• The attitudes and preferences of FPs about having
  research data presented
• How well speakers link therapeutic
  recommendations to research data

12
Methods

• 2 annual 3-day Family Medicine review conferences
• Halifax (Dalhousie University CME)
• 230 registrants
• 42 presentations (lecture and concurrent
  sessions)
• Ottawa (University of Ottawa CME)
• 209 registrants
• 38 presentations (lecture and concurrent
  sessions)
• 3 data collection methods for each conference

13
Methods Data Collection

• Questionnaire
• Registrants and Speakers
• Evaluate understanding of statistical terms
• Focus groups
• 1.5 hour with FPs from each site
• Determine preferences about types of statistics
  used
• Analysis of presentations
• PowerPoint files
• Video recording of presentations

14
Methods Data Analysis

• Questionnaire
• Descriptive statistics
• Focus groups
• Recorded and transcribed
• Content analysis
• Analysis of presentations
• PowerPoint files
• How research data presented slide counts
• Videorecording
• Linking of research data to recommendations
  Likert scales

Two researchers
15
Results Questionnaire

• Registrants (N121 29 response rate)
• 97 physicians 81 certificant of CFPC
• 51 male 49 female
• Avg. yrs practice 18.0
• 27 completed EBM workshop in last 10 years
• Speakers (N20 33 response rate)
• 19 specialists, 1 family physician
• 70 male 30 female
• Avg. yrs practice 18.0
• 30 completed EBM workshop in last 10 yrs

16
Results Questionnaire
17
Results Questionnaire (Registrants N121)
18
Results Questionnaire (Speakers N20)
19
Results Questionnaire
"How much emphasis should CME speakers place on
presenting research results when making
therapeutic recommendations?"
Percent responses
20
Results Focus group

• 2 focus groups
• 9 males 5 females
• Most preferred statistics
• Numbers needed to treat
• Inverse ARR
• Confidence Intervals

21
Results Focus group

• 2 focus groups
• 9 males 5 females
• Thats a pretty impressive statistic (RRR)
• Youd never see that in a drug trial (Inverse
  ARR)
• I responded to that in a different way it also
  reminded me of the responsibility we have to
  share this information with our patients (Happy
  faces table)
• Youre getting less and less enthusiastic about
  prescribing
• Most preferred statistics
• Numbers needed to treat
• Inverse ARR
• Confidence intervals

22
Results Focus group

• Trust in speakers and CME providers
• Trust that specialists can critically evaluate
  research
• Skepticism about specialist ability to interpret
  data
• Attend CME because specialists simplify data and
  make recommendations supported by research
• Trust CME offices to ensure programs present data
  accurately and completely
• Disclose conflict of interest

23
Recommendations from focus group

• Give a short stats refresher at the start of a
  conference
• Provide a 1-page handout on stats in the
  registration package
• Use consistent approach use the same format for
  graphs and stats
• Provide more research details when presenting
  newer areas of medicine, pivotal trials, or
  during drug sponsored company events

24
Preliminary results PowerPoints (N26)

• General terms
• Graphs
• Risk (event rate)
• Prevalence
• Absolute or relative not specified

• Absolute terms
• Absolute risk reduction or increase
• Number needed to treat or harm

• Relative terms
• Relative risk
• Relative RR
• Odds ratio
• Hazard ratio

95 confidence intervals
25
Preliminary results PowerPoints (N26)
Number of slides with . . .
26
Preliminary results Powerpoints (N26)
Number of slides with . . .
27
Preliminary results PowerPoints (N26)
Number of slides with . . .
28
Preliminary results Presentations (N26)
Number of slides with . . .
29
Preliminary results Videotapes (n19)
30
Conclusions

• Most FPs have low to moderate level of
  statistical comprehension
• Speakers have higher level of statistical
  comprehension than FPs
• FPs want recommendations to be supported with
  research
• FPs want speakers to critically evaluate the
  research
• Most speakers present little data in relative or
  absolute terms or with 95 CIs
• Focus group recommendations for CME providers
• Provide short review on statistics for FPs
• Develop consistent format for presentation of
  research data

31
Next Steps

• Provide CME for FPs to help them understand
  research data
• Develop format for presenting data that FPs find
  helpful and understandable
• Provide faculty development for speakers
• Add question to evaluation form re presentation
  of research data

32
Questions?