Title: The%20Science%20of%20Learning%20and%20the%20Virtual%20Anesthesia%20Machine:%20Benefits%20of%20"schematic"%20simulations%20in%20learning%20about%20complex%20systems%20Ira%20Fischler%20Simulation%20Faculty%20Learning%20Community%20May%202008
1The Science of Learning and theVirtual
Anesthesia MachineBenefits of "schematic"
simulations in learning about complex
systemsIra FischlerSimulation Faculty Learning
CommunityMay 2008
2Colaborators
- Sem Lampotang (Anesthesiology)
- Cynthia Kaschub (Psychology)
- David Lizdas (Anesthesiology)
- And for jump-starting this effort
- Sue Legg (Director, Partnership for Global
Learning)
3Plan for the talk
- Learning with understanding The idea of
mental models in psychology and education - How multimedia presentations can boost learning
- Potential advantages of simulation
- Transparent simulations and understanding
- The Virtual Anesthesia Machine (VAM)
- Learning with Transparent versus Opaque VAM
- Bridging abstract and concrete models Mixed
Reality and the Augmented Anesthesia Machine - A little bit about individual differences
4The mini-science of learning
- What makes a difference?
- Amount of practice (and the Power Law)
- Distribution of practice (and the Spacing Effect)
- Quality of practice (and Depth of Processing)
- Making the information distinctive
- Building appropriate mental models
Im doing great in all my other classes. I read
the book, came to class, outlined the material,
and made flash cards, and still got a C. Well,
did you understand the material? I thought I
did
5Mental models and schemas in comprehension
If the balloons popped, the sound wouldnt be
able to carry since everything would be too far
away from the correct floor. A closed window
would also prevent the sound from carrying, since
most buildings tend to be well insulated. Since
the whole operation depends on the steady flow of
electricity, a break in the middle of the wire
would also cause problems. Of course, the fellow
could shout, but the human voice is not loud
enough for the sound to carry that far. An
additional problem is that the string could break
on the instrument. Then there would be no
accompaniment to the message. It is clear that
the best situation would involve less distance.
6(No Transcript)
7Mental models in cognitive science
- Term first used by Kenneth Craik (43)
- If the organism carries a small-scale model of
external reality and of its own possible actions
within its head, it is able to try out various
alternatives, conclude which is the best of them,
react to future situations before they arise,
utilise the knowledge of past events in dealing
with the present and future, and in every way to
react in a much fuller, safer, and more competent
manner to emergencies which face it. (Craik, The
Nature of Explanation, 1943) - Quality of the model depends on how well it
captures the features of the domain that are
critical for the task at hand
8Understanding problems (Greeno, 1977)
- Our internal representation (or model) of the
problem should have - accurate CORRESPONDENCE between relevant elements
in the world and model - good COHERENCE between elements in the model
- appropriate links to PRIOR KNOWLEDGE that can aid
problem solving
correspondence
links to prior knowledge
coherence
mental model of problem
environment
9Little things (can) mean a lot(aka the devils
in the details)
- Subtle changes in problem framing can have
drastic effects on performance - Effects of analogy on solving the X-ray problem
- Preceded by bulb filament problem
- fragile glass framing 33 then solve X-ray
- laser intensity framing 69 then solve X-ray
- Effects of lives lost/saved on risky decisions
- Disease control programs, one more risky
- Lives saved framing 22 choose risky action
- Lives lost framing 75 choose risky action
10Pictorial Representations
- Came before text, historically
- Illustrations and drawings
- To illuminate structure, function and relations
- Animations and videos
- To make system dynamics visible
- Interactive simulations
- To actively explore cause-effect dynamics, test
hypotheses, etc. - Advantages of multimedia can be dramatic
11Mayers work on Multimedia (e.g., How
lightning forms)
- Compares..
- text-only to text-with-illustration (often
schematic) - Narration-only with narration-plus-animation
- Tests..
- Retention by free recall of presented facts
- Transfer (understanding?) by generating solutions
to - Redesign (how could you decrease lightning
intensity?) - Troubleshooting (how could there be clouds, but
no lightning?) - Prediction (what would happen with lower air
temperature?) - Abstraction of Principles (What causes
lightning?)
12Retention and transfer with MM
- Retention Modest MM gains
- Across 6 studies, 23 gain, 0.67 effect size
- Transfer Dramatic MM gains
- Across 6 studies, 89 gain, 1.50 effect size
13Potential advantages ofComputer-based simulation
- Cost cheap systems, easy to replace, low risk
- Track performance and provide just-in-time
feedback on performance - Virtually Real when needed
- But Reality can be played with
- Increase likelihood of rare but important events
- Increase salience of important features
- Present hyper-real depictions of space and time
- Make the abstract concrete, and the invisible
visible
14Instructional Choice-Points
- What do we want them to learn?
- Declarative knowledge, procedural skills
- Immediate or long-term retention
- Reproductive or creative, flexible learning
- How do we structure the learning?
- Amount of grounding in the domain
- Balance of guided (reception) and free
(discovery) learning - Amount of online assessment and intelligent
tutoring - Student-tailored, or one-size-fits-all
15Opaque versus Transparent Reality
- Opaque representation Simulation may be closely
analogous to the physical system (iconic,
concrete, high-fidelity, virtual reality) but
hides underlying structure, functions and
relations - Transparent representation Simulation sacrifices
physical fidelity but makes underlying aspects of
system overt (abstract, idealized, schematic,
symbolic)
16Transparency in simulations
- Hollans STEAMER (1981)
- Goldstones Concreteness Fading (2004)
- Butchers simplified diagrams (2006)
- Debate focusses on extent of fidelity and
whether detail helps or hurts - Little direct comparisonof simulation formats
17The Opaque-Reality VAM
18The Transparent-Reality VAM
19The Virtual Anesthesia Machine wide use, little
data
- 10 man-years of development time
- Available for free to individuals on the web
- Over 10,000 registered users
- Many positive reviews, both formal and informal
- Our goal assess the effectiveness of VAMs
Transparent Reality approach to simulation
20Training Session
- 30-page instructional guide developed
- Provides foundation of knowledge
- About anesthesia
- About the anesthesia machine and its subsystems
- Guided tour of several subsystems
- Breathing circuit
- Mechanical ventilation
- Manual ventilation
- Stresses visualization of dynamics using VAM
21Workbook Sample text
- Question 1 elimination of CO2. Are the gases
exhaled by a patient scrubbed of CO2 before
entering the bellows during mechanical
ventilation? - Demonstration using VAM Simulation
- ____ Click Reset to start simulation afresh
- ____ Point to the O2 flowmeter control knob to
enlarge it, then click-and-hold, and drag it
counterclockwise until the O2 bobbin inside is
about halfway up the tube.
22Workbook sample text (contd)
- What does this do? What happens to the flow of O2
from the supply line? - Opening the valve increases the flow of O2 from
the supply line into the breathing circuit. - Trace along its route through the plumbing. Where
does it wind up? - It depends. For example, If mechanical
ventilation is selected, but not on, the O2
flows backward through the CO2 absorber, past
the bellows and into the scavenger system
23Performance on Day 2 Testsundergraduate health
majors
24Performance on Day 2 Tests 2nd-year medical
students
25Performance on AAA Board Exam Review questions
(4AFC)
26Judgments about VAM
- Confidence Judgments
- On Component function
- Significantly higher for Transparent VAM (p lt
.01) - On System dynamics
- Marginally higher for Transparent VAM (p lt .15)
- Preferences for additional study
- 17 of 20 in Transparent group (UG) prefer TR VAM
- 11 of 20 in Opaque group prefer (UG) TR VAM
- 2 in TR, 7 in OR, think both would be preferable
to either - Similar trends among medical students more want
both
27Where to next?
- Combination and order effects
- Goldstones concreteness fading method?
- More precise tests of transfer
- Transfer to procedural skill does TR improve
error detection and response? - Hybrid simulations John Quarles project
28The Augmented Anesthesia Machine (AAM)
- Integrating transparent and realistic
representations with mixed-reality simulation
John Quarles and his Magic Lens
29Declarative and Procedural Knowledge with VAM and
AAM
- Two groups of undergrads
- Training
- Introduction to AM with VAM
- positioning components within the actual AM
- Five step-through exercises with VAM or AAM
- Day 2 Testing
- Declarative Board Exam Questions
- Procedural Find a machine fault in the AM
30Performance with VAM vs. AAM
31 Abstract and Concrete Knowledge
- Although the VAM may offer improved abstract
knowledge, participants found it difficult to
transfer this knowledge to the concrete
anesthesia machine. This is precisely the concern
that anesthesia educators have had with the VAM. - For example, many VAM participants understood the
abstract concept of the inhalation valve and they
correctly answered the written questions
regarding the gas flow in the valve. However,
during the fault test, they could not perform the
mental mapping between the abstract
representation of the VAM inhalation valve and
the concrete representation of the real
anesthesia machine inhalation valve. Thus, it was
difficult for VAM participants to apply their
abstract knowledge to a concrete problem, such as
the problem presented in the fault test.
32Role of Spatial Abilities?
- Three tests of spatial cognition
- Arrow-pointing working memory (small-scale)
- Perspective-taking (mid-scale)
- Navigating virtual environment (large-scale)
- Correlations of spatial abilities and performance
tend to be larger with VAM than AAM - Suggests those with strong visualization skills
can compensate for impoverished materials
33What weve learned
- Dynamic simulations can improve comprehension of,
and memory for, complex systems, BUT - - Different kinds of simulation are optimum for
different kinds of learning - So we need to know the goal of training
- Experience with both abstract (schematic) and
concrete (hi-fidelity) simulations may be optimum - So we may need an integrated approach
- Individual differences in domain-specific skills
and abilities will impact effectiveness of
representations - But we need to know how much
34It takes a Village (or at least a Learning
Community)
- Cognitive/human factors psychologists
- Usability analysts
- Instructional psychologists and educators
- Simulation designers and engineers
- Domain experts and professionals
35Thanks to all those RAs
- Emily McAlister
- Jonathan Greenwood
- Julianna Peters
- Shannon Bowie
- Sheila Holland
- Trudy Salmon