Kids, Cats and Concepts: Toward a Grand Unified Theory of Thinking

1
Kids, Cats and Concepts Toward a Grand Unified
Theory of Thinking

• Geoff Norman

2
The Goal

• To link research in three domains
• Dual processing models of thinking
• Exemplar and prototype models of categorization
  / concept formation
• Expertise and clinical reasoning
• to a greater understanding of human
  information processing

3
How I got there

• Distant
• Studies of clinical problem-solving
• Intermediate
• Role of experience in clinical reasoning
• Recent
• Diagnostic errors and dual processing

4
A Difficult Diagnostic Task
5
(No Transcript)
6
An Easy Diagnostic Task
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
What are all these things?
11

• chairs,
• (of course)

12

• What makes something a chair?
• What does the chapter on chairs in
• Harrisons Textbook of Internal Design
• look like?
• (What are the signs and symptoms of chairs)

13
The rule describes this.
14
Does it cover these?
15
(No Transcript)
16
(No Transcript)
17

• The rule is insufficient for the classification
  task
• But we can do the task quickly, accurately, and
  effortlessly
• HOW?

18
The Role of Similarity
19
Which is this most like?
20
Perceptual similarity - holistic
Perceptual Similarity The two objects share
common characteristics - similar colouring,
similar overall shape, wings / flippers, tail -
fin. They look the same.
Available without analytical knowledge Pattern
recognition, holistic processing
21
Causal similarity, conceptual
Conceptual Similarity The two objects share
common conceptual features (mammals) - live
birth, lactation, lungs, livers, etc.
Available only with specific analytical knowledge
Causal reasoning
22
Functional / Feature similarity
Function/feature Shared features - dorsal fin,
tailfin, body shape. Shared functio movement in
water, size, eat fish
They look a lot alike Size, shape, etc.
Available either from perceptual similarity or
individual features
23
Which is this most like?
24
Perceptual similarity - holistic
The two objects share common characteristics
- similar colouring, similar overall shape,
wings / flippers, tail - fin. Swim underwater.
Eat fish. Air breathing
Available without analytical knowledge Pattern
recognition, holistic processing
25
Conceptual similarity
The two objects share common conceptual
features (mammals) - live birth, lactation,
lungs, kidneys, etc.
Available only with specific analytical knowledge
Causal reasoning
26
DUAL PROCESSINGTwo basic strategies

• System 1
• Based on holistic similarity to prior examples
• Exemplar theory (more later)
• System 2
• Based on underlying conceptual characteristics
• Causal models

27
System 2 thinking

• Playing by the rules

28
(No Transcript)
29
Analytic View of Expertise

• The matters that set experts apart from
  beginners are symbolic, inferential, and rooted
  in experiential knowledgeExperts build up a
  repertory of working rules of thumb or
  heuristics that, combined with book knowledge,
  make them expert practitioners.
• E. Feigenbaum. The fifth generation artificial
  intelligence and Japan's computer challenge to
  the world. 1983

30
System 1 thinking

• Ive seen it before and here it comes again

31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34

• Successful categorization
• From 2-D abstract representation
• without analysis of features
• without language
• Successful generalization
• To other 2 D abstraction in atypical orientation

35
The Non- Analytic View

• We must be prepared to abandon the traditional
  view that runs from Plato to Piaget and Chomsky
  that a beginner starts with specific cases and
  abstracts and interiorizes more and more
  sophisticated rules.It might turn out that skill
  acquisition moves in just the opposite direction
  from abstract rules to particular cases. H.L.
  Dreyfus, 2002

36
Outline

• Dual processing
• Concept formation and categorization
• Expertise and Clinical Reasoning
• Applications -- Implications
• Levels of Processing
• Perception
• Transfer
• Aging and reasoning
• Intelligence

37
Three Literatures
Concept formation (categorization)
Dual Processing (Thinking)
Medin,Brooks
Clinical Reasoning
Stanovich, Evans, Kahnemann
Norman,Schmidt
38
Dual Processing

• System 1
• Rapid, unconscious, based on concrete
  similarity, just pattern recognition
• System 2
• Slow, logical, conceptual, energy-intensive,

39
CHARACTERISTICSSystem 1 System 2

• Unconscious
• Implicit
• Automatic
• Effortless
• Rapid
• Holistic,
• Old (evolution)
• Contextualized

• Conscious
• Explicit
• Controlled
• Effortful
• Slow
• Analytic
• New (evolution)
• Abstract

40
Neuroanatomy of System 1,2

• System 1
• right inferior prefrontal cortex
• Evans, 2008
• Involves hippocampus
• Smith DeCoster, 2000

• System 2
• ventral medial prefrontal cortex

41
Neurophysiology of System 1,2

• Glucose dose (vs. Placebo)
• Shift of processing strategy toward System 2
  (more energy demand) with glucose load
• (Attraction effect - 17 vs. 47)
• (Masicampo Baumeister, 2008)

42
Mental representations

• SYSTEM 1
• Abstract concepts
• Feature list, probability, causal mechanism,
  process
• SYSTEM 2
• ???????

43
Categorization / Concept Formation
44
Prototype Theory - Rosch, Mervis

• Categories are defined by an abstracted prototype
• Prototype has most features of category, fewest
  features of confuseable categories (Bird--gt
  robin)
• identification of category membership based on
  number of features in common with prototype
• Retrieval process is analytic, hence based on
  relevant features
• Retrieval process is deliberate, logical, slow
• (Like System 2)

45
Exemplar Theory - Medin, Brooks

• Categories consist of a collection of prior
  instances
• identification of category membership based on
  availability of similar instances
• Retrieval process is non-analytic (unaware),
  hence can result from objectively irrelevant
  features
• Retrieval process is not deliberate, not
  available to introspection
• (Like System 1)

46
Dual Processing in Medicine

• From Process to Knowledge
• (Analytical and Experiential)

47
The beginnings - clinical reasoning as a process

• Hypothetico-deductive method
• (Elstein, Shulman, Sprafka, 1977)
• Expert (and novice) clinicians generate multiple
  diagnostic hypotheses early in the encounter then
  gather data to confirm (usually) these hypotheses

48
Does hypothesis predict accurate solution?
Barrows, Neufeld, Norman, 1981
49
Where do hypotheses come from?

• Medical experts differed from novices in that
  they generated better hypotheses
• and we dont know why!
• A. Elstein
• Dx Error Conference
• May 31, 2008

50
Expert Physicians and Dual Processing

• To what extent does the
• formal knowledge of medical school
• vs.
• experiential knowledge of practice
• contribute to expertise

51
Schmidt Norman, 1991
Novice Intermediate Expert
Examples
Basic Science Mechanisms
Clinical Rules
System 2
System 1
Basic Science Mechanisms
Clinical Rules
Basic Science Mechanisms
52
Who do you pick?

• Dr. JW completed the specialty exam last year and
  stood 14th in the country.
• Dr. WS completed the specialty exam 6 years ago.
  At the time, she was in the top 1/3 of all
  candidates.

53
The Conundrum

• Why do we prefer the candidate with apparently
  less competence but much more experience?
• What did she get from 10 years of experience?
• 10 years of experiences
• (System 1 knowledge)

54
BUT

• Every measure of formal (System 2) knowledge
  decays right after graduation

55
Day and Norcini, 1988
56
RESEARCH AGENDAThe Role of Experience

• In the course of becoming an expert, one
  requires an extensive stable of examples which
  guide diagnosis and management of new problems
• How is this experiential knowledge stored and
  retrieved from memory?
• How is this knowledge coordinated with analytical
  knowledge?

57
Evidence of System 1 in Diagnostic Reasoning
58
Visual Diagnosis and Response Time

• STUDY
• 100 slides in 20 categories
• Students, clerks, residents, GPs,
  Dermatologist
• Accuracy and Response Time

59
Accuracy by Educational Level
60
Response time by Educational Level
61
Evidence of Exemplars
62
Effect of Similarity (Allen, Brooks, Norman,
1992)

• 24 medical students, 6 conditions
• Learn Rules
• Practice rules
• Train Set A Train Set B
• (6 x 4) x 5 (6 x 4) x 5
• Test (9 / 30)

63
(No Transcript)
64
(No Transcript)
65
Accuracy by Bias Condition
66

• Is it just visual similarity?
• If its non-analytic does it apply to
  objectively irrelevant features?

67
ECG InterpretationHatala et al, 1999

• Medical students/ Fam Med residents
• PRACTICE (4/4 7 filler)
• middle aged banker with chest pain
• OR
• elderly woman with chest pain
• Anterior M I
• TEST ( 4 critical 3 filler)
• Middle aged banker with chest pain
• Left Bundle Branch Block

68
RESULTSPercent of Diagnoses by Condition
69
CONCLUSIONS - Medical Diagnosis and Dual
Processing

• Experiential knowledge is a major contributor to
  diagnostic expertise
• Categories and concepts are based on our specific
  experience with the world
• These specific experiences are accessed and used
  without awareness

70
When do experts use system 2?
71
Analytic reasoning and Diagnosis

• Invoked for confirmation in all Dx encounters
• Analytic knowledge of many forms
• Illness scripts
• Symptom-disease probabilities
• Semantic axes
• Feature lists (e.g. DSM 4)

72
Where Do Clinicians Use Basic Science?

• Most use basic science rarely?
• Observational studies (Schmidt, Patel)
• Some use basic science some of the time
• Difficult problems in nephrology
• Some use physiology ALL the time
• Intensivists, anesthesiology

73
Most use it rarely(Patel, Schmidt)

• Clinicians rarely use basic science explanation
  in routine practice.
• While they may possess the knowledge, it remains
  encapsulated until mobilized for specific goals
  (to solve specific problems) (Schmidt, HG)

74
Some Use it with Difficult Cases(Norman, Brooks,
Trott, Smith)

• When experts are confronted with difficult cases,
  do they revert to causal reasoning?

75
Experimental Design

• R1 --GP R2 -- IM Nephrol
• n4 n4 n4
• Clinical Cases
• k 8
• Explain and Diagnose

76
Diagnostic Accuracy
77
Causal Explanations
78
No of Diagnoses / Investigations
79
Conclusions - Use of Basic Science

• In difficult diagnostic situations, clinicians
  use causal physiological knowledge and analytic
  reasoning
• Expertise associated with more coherent
  explanations, better diagnosis

80
Dual Processing and Experience

• With increasing experience, do people rely more
  or less on System 1 -- Non-analytic reasoning?

81
Studies of Relative Experts(Moruzi, Brooks,
Norman, 2003)

• Dermatologists/ GPs / residents
• 36 slides (typical / atypical)
• Condition A
• Verbal description of slide (verbal)
• then photo (visual verbal)
• Condition B
• Photo only (visual)

82
Diagnostic Accuracy
83
Diagnostic Accuracy
84
Hatala et al.

• ECG Diagnosis
• Prior match / unmatch history
• Postgraduate residents and med students

85
RESULTSPercent of Diagnoses by Condition
Medical Students
86
Dual Processing and Instruction
87
Role of Instruction in reasoning

• Since NA (System 1) reasoning occurs at all
  levels, is effective, is automatic
• You cant
• tell student to not do it
• tell student to beware of biases
• tell student to think of better diagnoses

88
Does a coordinated strategy improve accuracy?

• Norman, Brooks, Colle (ECG)
• Schmidt and Mamede (Gen Medicine)
• Ark Eva, (ECG)

89
Norman, Brooks Colle, 2000

• Contrast instructions to
• Think of the first thing that comes to mind, then
  consider features
• vs.
• Gather all the data then arrive at diagnosis
• 32 Undergrad Psychology students
• 11 disorders, rules examples
• Test -- 10 new ECGs

90
Diagnostic Accuracy
Pattern Rules Rules
91
Schmidt Mamede, 2005

• 42 I.M. residents
• 16 written cases --- simple / complex
• Within subject/case design
• Instructions
• First thing that comes to mind
• vs.
• Hypotheses, findings for/against, differential,
  .

92
Diagnostic Accuracy
93

• Does increased reliance on pattern recognition
  with experience have a cost?

94
Moruzi, Norman, Brooks, 2005
Subjects 39 medical students in McMaster MD
Programme 3rd instructional unit (7 months
completed). No prior training in dermatology  
Materials Learn 10 cases Test on 20 new cases,
matched / unmatched on similarity Instructions
First thing then Carefully consider
features vs. Carefully list features and
diagnose
95
Combined vs. Individual Strategies
96
ECG Diagnosis - Ark Eva

• 48 undergrad psychology students
• 8 ECG diagnoses (A/A, B/B, C/C,D/D)
• Instructions
• Compare and contrast vs. Sequential
• Combined Analytical/Non-analytical vs.
• usual approach
• Test
• 20 ECGs (10 old, 10 new)
• Immediate / 1 week later

97
Effect of Examples and Instructions on New Cases
after One Week
Ark Eva, 2005
98
Conclusions - Dual Processing and Diagnosis

• Evidence that clinicians access both kinds of
  knowledge/ use both processes
• Evidence that with increasing experience, greater
  reliance on system 1
• Evidence that students benefit from explicit
  instruction to use both

99
Dual Processing and Thinking

• DP and levels of processing
• DP and perception
• DP and transfer
• DP and aging

100
Dual Processing and Levels of Processing

• Are conceptual structures and deep processing an
  underpinning to development of System 1
  (non-analytic) knowledge?
• (Dreyfus)

101
Role of Basic Science in Novice Reasoning(Woods,
Brooks, Norman, 2003)

• 4 neurology / muscular diseases
• 36 medical students
• Basic Science or Symptom/Disease probability

102
Measurement

• Diagnostic Test
• 15 cases, 4-6 features
• Administered at 0, 7 days

103
Score on Dx Test
104
Score on Dx Test
105
Score on Dx Test
106

• Basic science is used to construct and
  reconstruct coherent relations between symptom
  and disease

107
Dual Processing and Perception
108
Word Superiority Effect

• Higher - level concepts (words) in memory
  facilitate recognition of elements of words and
  pseudo-words
• R I N K
• B I N K
• N R I K
• - possibly because of rapid (top-down) then slow
  (bottom up) processing

109
Influence on Feature Interpretation

• Diagnostic hypotheses arise from pattern
  recognition processes based on similarity to
  prior examples
• In situations of feature ambiguity, hypotheses
  may influence what is seen

110
Influence of Diagnosis on Feature Perception
(LeBlanc et al)

• 20 residents, 20 final year students
• 8 photos of classical signs from clinical
  diagnosis textbooks
• Correct history and diagnosis
• vs.
• Incorrect history and diagnosis

111
(No Transcript)
112
RESULTSDiagnostic Accuracy by Bias
113
RESULTSNumber of Features of Correct Diagnosis
by Condition
114
RESULTSNumber of Features of Alternate Diagnosis
by Condition
115
Dual Processing and Transfer

• Although medical (and other) study is directed at
  conceptual learning, use of conceptual knowledge
  to solve problems (transfer) is rare and
  difficult.
• WHY????

116
Spontaneous Transfer

• 8 high performing undergrad (Health Sciences)
  students.
• 3 principles (Laplace, Poiseuille, Starling)
• 12 test cases
• Score
• 0 wrong answer,
• 1 right answer, wrong explanation
• 2 right answer, right but poor explanation
• 3 right answer, good explanation

117

• Laplaces Law
• In a cylindrical vessel, the wall tension is
  proportional to the radius and pressure exerted
  by the vessel contents. This can be expressed as
  T PR where T is wall tension, P is pressure
  exerted by the contents, and R is the radius of
  the vessel.

118

• A 72 year old female has been diagnosed with an
  aneurysm (dilatation) of the aorta. The doctor
  tells her that if it grows to 5 cm in diameter
  she will need surgery to prevent bleeding.
  Explain why the increasing diameter is a problem.

119
Average Score

• 15.2/36 42

120

• during early learning, the principle is only
  understood in terms of the earlier example the
  principle and example are bound together. Even if
  learners are given the principle or formula, they
  would use the details of the earlier problem in
  figuring out how to apply that principle to the
  current problem (Ross, 1987)

121
Why are the examples so seductive?

• System 1
• Fast, unconscious, contextualized, concrete
• System 2
• Slow, logical, abstract
• Transfer amounts to overriding System 1 to
  utilize abstract, conceptual information

122
DP and Age

• Evidence from psychology that with increasing
  age, we rely more on System 1 thinking

123
Eva Cunnington, 2006

• 15 family docs, 7 lt 60 yr., 8gt60 yr.
• 8 cases
• 2 diagnoses, 4 conditions
• Generated, Provided, Privileged, Extreme
• ---------------gtgtgtgtgtgt weight on second diagnosis

124
Diff (Dx 1 - Dx2)
125
DP and Intelligence

• The Flynn Effect
• If the test is held constant, every generation
  scores about 8 IQ points higher than previous.
• Why??

126

• Vocabulary, math
• - constant over generations
• Similarities
• A mouse is to a horse as a minnow is to a
• Whale
• Shark
• Tuna
• Elephant
• large generational, cultural differences

127

• Flynn
• - Generational differences reflect increasing
  use of concepts in society
• - Nurture, not nature

128
Gardner, Sternberg

• Triarchic theory of intelligence -
  Sternberg
• Multiple intelligences - Gardner
• People can be very intelligent in their own
  domain but cannot perform on abstract tasks in IQ
  test

129
Additional evidence

• I.Q. is strongly related to size of working
  memory
• System 2 loads heavily on working memory
• System 1 independent of working memory

130

• Does conventional IQ (or some aspects) reflect
  System 2?
• Contextually imbedded intelligence reflect System
  1

131
Some Last Words

• (from other people)

132

• the expert does not solve problems. He does not
  even think. He just does what normally works,
  and, of course, it normally works. The expert is
  simply not following any rules! He is
  discriminating thousands of special cases.
• H Dreyfus

133

• In general, to preserve expertise we must foster
  intuition at all levels of decision-making,
  otherwise wisdom will become an endangered
  species of knowledge.
• H. Dreyfus

134
First and Last Word on Expertise

• It is a profoundly erroneous truism, repeated
  by all copy-books and by eminent people making
  speeches, that we should cultivate the habit of
  thinking about what we are doing. The precise
  opposite is the case. Civilization advances by
  extending the number of operations which we can
  perform without thinking about them. Operations
  of thought are like cavalry charges in a battle
  -- they are strictly limited in number, they
  require fresh horses, and must only be made at
  decisive moments.
• A.N. Whitehead, 1911 (in J Bargh, 1999)

135
Bibliography

• Dreyfus HL From Socrates to expert systems The
  limits and dangers of calculative rationality.
  http//socrates.berkeley.edu
• Evans J St BT. In two minds dual - process
  accounts of reasoning. Trends in Cognitive
  Science 2003 7 454-459
• Evans J StBT. Dual processing accounts of
  reasoning, judgment and social cognition. Ann Rev
  Psychol 200859 255-78.