Teaching for Engagement and Retention: Methods to Increase Understanding of Science and Improve Patient Care

1
Teaching for Engagement and RetentionMethods to
Increase Understanding of Science and Improve
Patient Care

• Molly Cooke MD
• Professor of Medicine, UCSF
• Senior Scholar,
• Carnegie Foundation for the Advancement of
  Teaching

2
My highest goal for my IAS-USA lectures is to

1. Increase learner knowledge
2. Advance learner skills
3. Change the clinical behavior of learners
4. Improve clinical outcomes for HIV patients

ARS 1
3
Overview of Talk

• Review the effectiveness of traditional CME
  lectures
• How practicing physicians (and everyone else)
  learn
• Promoting significant learning in large group
  settings
• Suggestions for IAS-USA core faculty and CME
  planners

4
Overview of Talk

• Review the effectiveness of traditional CME
  lectures
• How practicing physicians (and everyone else)
  learn
• Promoting significant learning in large group
  settings
• Suggestions for IAS-USA core faculty and CME
  planners

5
Characteristics of CME lectures include

1. Learners are optimally attentive only for the
   first five minutes
2. Most learners are no longer concentrating after
   20 minutes
3. Lectures are minimally effective in increasing
   knowledge
4. Knowledge associated with attending a CME lecture
   has reverted to baseline after a month
5. Knowledge associated with attending a CME lecture
   has reverted to baseline after a year

ARS 2
6
Im changing a bit how Im doing it just to try
to learn differently. Lectures are good for me
in the sense of getting an overview, but it
doesnt stick with me very longWhat sticks with
me best is what I do in practice. I see a
problem and I have to go to the books and figure
out what to do for somebody. (Physician 5)
Learning in Practice Experiences and Perceptions
of High-Scoring Physicians Sargeant Acad Med 2006
7
Impact of ARS on initial and one month learning
Quiz score by lecture type
Quiz score (best7) N P value (1 month difference) P value (vs. other lecture types)
Basic lecture Initial 4.250.28 32 .05 ----
One month 3.390.33 18 ----
Interactive lecture Initial 6.500.1 22 lt .001 lt .001 (vs. basic initial ) .31 (vs. ARS initial)
One month 4.220.37 18 .11 (vs. basic 1 month) .11 (vs. ARS 1 month)
ARS (May - Aug 02) Initial 6.700.13 23 lt .001 lt .001 (vs. basic initial )
One month 4.670.45 12 lt .05 (vs. basic 1 month)
ARS (Sept 02- Jan 03 Initial 6.560.19 27 lt .001 ----
One month 5.070.34 14 ----
Shackow 2004 Family Medicine
8
Impact of ARS on initial and one month learning
Quiz score by lecture type
Quiz score (best7) N P value (1 month difference) P value (vs. other lecture types)
Basic lecture Initial 4.250.28 32 .05 ----
One month 3.390.33 18 ----
Interactive lecture Initial 6.500.1 22 lt .001 lt .001 (vs. basic initial ) .31 (vs. ARS initial)
One month 4.220.37 18 .11 (vs. basic 1 month) .11 (vs. ARS 1 month)
ARS (May - Aug 02) Initial 6.700.13 23 lt .001 lt .001 (vs. basic initial )
One month 4.670.45 12 lt .05 (vs. basic 1 month)
ARS (Sept 02- Jan 03 Initial 6.560.19 27 lt .001 ----
One month 5.070.34 14 ----
Shackow 2004 Family Medicine
9
Impact of CME vs. a Control Group by
Intervention Type and Intensity
Davis JAMA 1999
10
Impact of CME vs. a Control Group by
Intervention Type and Intensity
Davis JAMA 1999
11
Overview of Talk

• Review the effectiveness of traditional CME
  lectures
• How practicing physicians (and everyone else)
  learn
• Promoting significant learning in large group
  settings
• Suggestions for IAS-USA core faculty and CME
  planners

12
Please make your selection...

1. Choice One
2. Choice Two
3. Choice Three
4. Choice Four
5. Choice Five

13
How Outstanding Physicians in Practice Maintain
Competence

• Multisource assessment of family physicians in
  Nova Scotia
• Six performance domains patient communication,
  team communication, professionalism, clinical
  performance, office management
• 142 physicians evaluated by 8 medical colleagues,
  8 co-workers, 25 randomly selected patients

Sargeant Acad Med 2006
14
How Outstanding Physicians in Practice Maintain
Competence - 2

• Twenty-five highest scoring physicians invited to
  be interviewed
• Findings Informal learning fundamental
• Prompted by patient problems
• Supported by interactions with colleagues

Sargeant Acad Med 2006
15
If youre a very busy clinician you know where
youre a little bit unsure of yourself, you know
where you might be just a little bit shaky
sometimes, thats the kind of stuff you look for,
and I think that the awareness just comes out of
your own work. (Physician 9)
Im interacting on a daily basis with
colleagues, family doctors, consultants. Its a
very valuable aspect of practice and really CME.
(Physician 8)
Learning in Practice Experiences and Perceptions
of High-Scoring Physicians Sargeant Acad Med 2006
16
Promoting significant learning in large group
settings

• Learner identification of content
• High level of learner interaction
• Multiple modes of learning
• Sustained engagement

17
Impact of ARS on initial and one month learning
Quiz score by lecture type
Quiz score (best7) N P value (1 month difference) P value (vs. other lecture types)
Basic lecture Initial 4.250.28 32 .05 ----
One month 3.390.33 18 ----
Interactive lecture Initial 6.500.1 22 lt .001 lt .001 (vs. basic initial ) .31 (vs. ARS initial)
One month 4.220.37 18 .11 (vs. basic 1 month) .11 (vs. ARS 1 month)
ARS (May - Aug 02) Initial 6.700.13 23 lt .001 lt .001 (vs. basic initial )
One month 4.670.45 12 lt .05 (vs. basic 1 month)
ARS (Sept 02- Jan 03 Initial 6.560.19 27 lt .001 ----
One month 5.070.34 14 ----
Shackow 2004 Family Medicine
18
Promoting significant learning in large group
settings

• Learner identification of content
• High level of learner interaction
• Multiple modes of learning
• Sustained engagement

19
Writing Good MCQs - Review

• Use short stem
• Avoid cueing
• Repetition of word in the correct answer that
  occurs in the stem
• Longest answer is correct
• Make sure options are homogeneous
• Functional test for good MCQ - item can be
  answered with opotion list covered

20
MCQ Flaws

• Characteristics of CME lectures include
• Learners are optimally attentive only for the
  first five minutes
• Most learners are no longer concentrating after
  20 minutes
• Lectures are minimally effective in increasing
  knowledge
• Knowledge associated with attending a CME lecture
  has reverted to baseline after a month
• Knowledge associated with attending a CME lecture
  has reverted to baseline after a year

• The MCQ to the left illustrates the flaw of
• High conceptual challenge
• Over-lengthy stem
• Non-homogeneity
• Cueing

ARS 3
21
MCQ Flaws

• Characteristics of CME lectures include
• Learners are optimally attentive only for the
  first five minutes

• Most learners are no longer concentrating after
  20 minutes
• Lectures are minimally effective in increasing
  knowledge
• Knowledge associated with attending a CME
  lecture has reverted to baseline after a month
• Knowledge associated with attending a CME
  lecture has reverted to baseline after a year

The MCQ to the left illustrates the flaw of

1. High conceptual challenge
2. Over-lengthy stem
3. Non-homogeneity
4. Cueing

ARS 3
22
Characteristics of CME lectures include

1. Learners are optimally attentive only for the
   first five minutes
2. Most learners are no longer concentrating after
   20 minutes
3. Lectures are minimally effective in increasing
   knowledge
4. Knowledge associated with attending a CME lecture
   has reverted to baseline after a month
5. Knowledge associated with attending a CME lecture
   has reverted to baseline after a year

23
Using the ARS to Promote Engagement

1. To manage interaction start discussion, require
   interaction with peers, conduct votes
2. To get feedback from learners - demographics,
   response to presentation
3. For formative assessment e.g. apply lecture
   material to new situation
4. To do practice problems
5. To guide thinking, review, and lead through
   multistep process
6. To make lecture fun

24
Using the ARS and MCQs to Promote Conceptual
Understanding

• Well-written questions required as usual
• High conceptual challenge questions /-
  scaffolding
• Flexible WRT answers
• No correct answer, multiple partially correct
  answers, several correct answers all okay

25
(No Transcript)
26
Theory of Reasoned Behavior/Action
27
Knowledge and Dispositions

• Propositional or conceptual knowledge
• enables complex problem-solving tasks through an
  understanding of the likely nature of the problem
  and its associations with other, related
  considerations.

• Procedural knowledge
• How to achieve specific goals or tasks
• How to select approach
• How to strategically organize activities

• Dispositions The significance of
  dispositions lies in the differences between what
  individuals are capable of doing and what tasks
  they actually undertake.

Stephen Billett Learning in the Workplace -
Strategies for Effective Practice 2001
28
Which learner attribute is the target of the
numbered CME intervention?
Learner attributes

1. Propositional knowledge
2. Level 1 procedural knowledge
3. Level 2 procedural knowledge
4. Level 3 procedural knowledge
5. Learner dispositions

Theory of reasoned action and behavior
ARS 4, 5 and 6
29
Which learner attribute is the target of the
numbered CME intervention?
Learner attributes

1. Propositional knowledge
2. Level 1 procedural knowledge
3. Level 2 procedural knowledge
4. Level 3 procedural knowledge
5. Learner dispositions

Theory of reasoned action and behavior
ARS 4, 5 and 6
30
Which learner attribute is the target of the
numbered CME intervention?
Learner attributes

1. Propositional knowledge
2. Level 1 procedural knowledge
3. Level 2 procedural knowledge
4. Level 3 procedural knowledge
5. Learner dispositions

Theory of reasoned action and behavior
ARS 4, 5 and 6
31
Using ARS to Promote Peer Discussion
Nichol and Boyle Stud Higher Educ 2003
32
Overview of Talk

• Review the effectiveness of traditional CME
  lectures
• How practicing physicians (and everyone else)
  learn
• Promoting significant learning in large group
  settings
• Suggestions for IAS-USA core faculty and CME
  planners

33
Sociocultural theories of learning

• Learner identification of content
• High level of learner interaction
• Multiple modes of learning
• Sustained engagement

34
Recommendations for IAS-USA CME Presentations

• Increase learner control of topic and content
• Use ARS to promote learner interaction
• Strive to have multilevel educational
  interventions
• Encourage formation of learner communities of
  practice