Critical Thinking in Medical Education: Assessing What We Mean and What We Know - PowerPoint PPT Presentation

1 / 32
About This Presentation
Title:

Critical Thinking in Medical Education: Assessing What We Mean and What We Know

Description:

Critical thinking is... state what a poor critical thinker would do or say ... Critical Thinking as a Process ... – PowerPoint PPT presentation

Number of Views:44
Avg rating:3.0/5.0
Slides: 33
Provided by: edkr9
Category:

less

Transcript and Presenter's Notes

Title: Critical Thinking in Medical Education: Assessing What We Mean and What We Know


1
Critical Thinking in Medical EducationAssessing
What We Mean and What We Know
  • Ed Krupat, PhD
  • Director, Center for Evaluation
  • Medical Education Grand Rounds
  • Dec. 5, 2008

2
Components of Talks that I Have Liked
  • 1. A bit of context or framing
  • 2. A touch of theoretical/conceptual background
  • 3. A good portion of research and data
  • 4. A bounty of concern for practical application
  • 5. A strong dose of provocative discussion

3
Context The Origins of These Efforts
  • Derek Bok--to--Richard Hersh
  • Evaluation of programs
  • Confusion-to-clarity-to-confusion-to-???
  • Why is critical thinking so important
  • In life
  • In medicine
  • Desire to be data-driven

4
Conceptual Analysis Just What Is Critical
Thinking??
  • Overlapping concepts
  • Analytic reasoning
  • Problem solving
  • Decision making
  • Clinical/diagnostic reasoning/judgement
  • Habits of mind
  • Meta-cognition
  • Adaptive expertise

5
Some DefinitionsCritical thinking is
  • ...the intellectually disciplined process of
    actively and skillfully conceptualizing,
    applying, analyzing, synthesizing, and/or
    evaluating information gathered from, or
    generated by, observation, experience,
    reflection, reasoning, or communication as a
    guide to belief and action. (Scriven, 1996)
  • the art of thinking about your thinking while
    you are thinking in order to make your thinking
    better more clear, more accurate, and more
    defensible. (Paul et al, 1989)

6
Blooms Taxonomy
7
Research Data
  • Two projects being conducted simultaneously
  • 1.Qualitative Do physicians agree about just
    what critical thinking is?
  • 2.Quantitative Can we assess critical thinking
    among medical students?

8
What is Critical Thinking The Responses of
Physicians
  • Survey of practicing MD faculty at 5 medical
    schools
  • HMS, UCSF, Case Western, U Mass, Baylor (total
    n73)
  • Convenience sample

9
Two Tasks
  • Define critical thinking
  • Think of a clinical scenario in which critical
    thinking was important
  • describe it
  • state what a good critical thinker would do or
    say in that situation
  • state what a poor critical thinker would do or
    say
  • state how the outcome would differ if one or the
    other would have been the physician involved

10
Coding of the Definitions. 1
  • Definition specifically included
  • Collection of information/data 27
  • Making sense of information 96
  • Utilization for decision making 74
  • Utilization for action 14
  • Specific ties to medical context 43
  • Necessity of building upon knowledge base 27

11
Coding of the Definitions. 2
  • Critical thinking characterized as
  • A process
  • A skill or ability
  • A disposition

12
Critical Thinking as a Process
  • a process of reflective reasoning that uses
    objective evidence, a deliberate weighing of
    options and alternatives, and clinical judgement
    and experience to guide decision making.
  • the process by which one is able to rationally
    acknowledge different choices, processes and
    outcomes in the clinical encounter.
  • a process in which problems are being analyzed
    from different angles and connected to
    pre-existing knowledge before any conclusions are
    being drawn
  • 44 of all responses

13
Critical Thinking as a Skill or Ability
  • the ability to think through a problem using
    reasoning. Also the ability to judge the
    credibility of sources.
  • the ability to rigorously weigh the validity of
    evidence and then to effectively synthesize this
    evidence to reach a clinical decision.
  • the ability to effectively problem solve using
    known data or under conditions of uncertainty.
  • 50 of all responses

14
Critical Thinking as a Disposition
  • careful attention to what you know, vigilance
    for what you do not, and the courage to question
    both of the above categories.
  • thinking about an topic, issue, or challenge in
    a way that sets aside my immediate gut
    response,so that I can be open and reflective to
    other possible ways of viewing the challenge
  • thinking deeply, keenly, flexibly, openly,
    reflectively, with an awareness of self and
    others, with attention to what is known and
    unknown, and with humility.
  • 10 of all responses

15
Breakdown by Specialty
16
Clinical Situations An Interim Report
  • Looking for
  • What situations, choices, challenges
    differentiate critical and non-critical thinkers
  • What is it that critical thinkers do or say
  • What is it that non-critical thinkers do or say
  • What outcomes differ when critical thinking is or
    is not in evidence
  • Bottom line
  • What are the key differences in the thoughts,
    words, and actions of critical thinkers

17
Clinical Contexts and Tasks
  • Not surprising
  • Diagnosis and treatment
  • Surprising
  • Collaboration and interpersonal issues
  • How to protect patients rights and autonomy
  • How to deal with difficult patient requests
  • How to provide patient with appropriate options

18
The Differences
  • Critical thinkers
  • Do more data gathering
  • Avoid premature conclusions
  • See inconsistencies in information
  • Utilize knowledge more extensively and explicitly
    to make decisions
  • Are aware of limitations and doubts
  • Monitor and evaluate their own decisions
  • Involve patients more fully
  • Provide patients with options

19
A Few Conclusions, Many Questions
  • Physicians are not necessarily all speaking the
    same language when they discuss critical thinking
    with one another
  • Is critical thinking a skill, a process, or a way
    of looking at the world?
  • Are skill and disposition both necessary?
  • Do you teach/encourage/foster skills in the same
    way as dispositions???
  • Do you assess skills in the same way that you
    assess dispositions?
  • Where does building upon a knowledge base come in?

20
How do we assess critical thinking?
  • Collegiate Learning Assessment (CLA)
  • Used in over 200 colleges and universities
  • Assesses higher order outcomes
  • Students work on 90 minute performance tasks
  • System has been devised to generate reliable
    scores
  • Question Is this a valid measure of critical
    thinking for medical students?

21
Catfish One of the Performance Tasks Used
  • Grotesquely mutated catfish has been found in the
    local lake that supplies the towns water
  • You will serve on the mayors advisory panel
  • You are provided with 6 documents to read
  • Newspaper article
  • Editorial by environmental activist
  • Radio interview with a biologist from a nearby
    college
  • State report on water testing from lake
  • Area map
  • Journal article about similar discoveries

22
Task
  • Open-ended written questions ask students to
  • Identify main hypotheses to explain phenomenon
  • Identify strengths and weakness of each
  • State and defend most likely explanation
  • Suggest course of action

23
Scoring
  • Evaluation of evidence
  • What is relevant, what is valuable
  • Analysis and synthesis of evidence
  • Connections, inconsistencies, flaws in reasoning
  • Conclusion drawing
  • Acknowledging alternatives and options
  • Presentation of arguments
  • Concise, evidence-based, logically structured

24
Research Design
  • Recruit multiple schools
  • Recruit and compare students at two (or more)
    points in time
  • Entering Year 1 students took two tasks
  • Students at end of Year 3 took one task
  • Look for associations between CLA scores and
    other performance indicators
  • MCAT scores
  • Gender
  • Age
  • English as first language
  • Step scores
  • Compare scores between first and third year
    students

25
Analysis Problems
  • Among year 1 students
  • scores on the two tasks not as highly correlated
    as would be expected
  • Brain Boost task always given first
  • Year 1 students spend consistently less time and
    do consistently worse on Catfish (always second
    task) than on Brain Boost
  • Among Year 3 students--who only took one task,
    either Brain Boost or Catfish--students take more
    time and perform better than on Brain Boost
  • Small ns

26
Findings 1.
  • Medical school students have good critical
    thinking skills at baseline
  • Across the 3 schools, year 1s
  • fell at the 87th percentile among graduating
    seniors nationally
  • were 1.25 standard deviations above the national
    mean
  • Is that good news or, with selectivity of the
    medical schools, should we expect to be even
    higher??

27
Table 1. CLA performance by task, year, and school
plt.01
28
Table 2. Correlations among tests and
demographic variables
Significance codes plt.05, plt.01,
plt.001 Note Significance not adjusted for
multiple significance tests
29
Findings to Come
  • What is the correlation between CLA scores and
  • Step II CK
  • Step II CS
  • HMS Comprehensive Exam

30
What have we learned?
  • Critical thinking has many dimensions, and
    physicians may be talking past one another if
    they dont share meaning when they speak.
  • Critical thinking can manifest itself in how to
    deal with patients, not just in generating
    diagnoses and treatment plans.
  • Critical thinking involves being
  • Sensitive
  • Inclined
  • Able
  • (Perkins Ritchhart
    2002)

31
What have we learned? 2
  • Medical students enter school with solid critical
    thinking skills
  • Critical thinking is somewhat, but not greatly,
    associated with other measured outcomes
  • It is still unclear the extent to which medical
    education fosters or hinders critical thinking
  • We have more questions at the end of this process
    than we did than when it began, but the questions
    are
  • more sophisticated
  • more tied to possible action

32
Some Big Questions
  • If we think that critical thinking is so
    important, why dont we explicitly design the
    curriculum to assure that we teach it?
  • If we wanted to do so, how and when would it be
    taught?
  • Should measures of critical thinking, as a skill
    or disposition, be built into our admissions
    screening process?
  • Should critical thinking be a core competency
    that is assessed throughout the curriculum? as a
    graduation requirement? As a requirement for
    licensure?
Write a Comment
User Comments (0)
About PowerShow.com