The%20Science%20of%20Learning%20and%20the%20Virtual%20Anesthesia%20Machine:%20Benefits%20of%20"schematic"%20simulations%20in%20learning%20about%20complex%20systems%20Ira%20Fischler%20Simulation%20Faculty%20Learning%20Community%20May%202008

1
The Science of Learning and theVirtual
Anesthesia MachineBenefits of "schematic"
simulations in learning about complex
systemsIra FischlerSimulation Faculty Learning
CommunityMay 2008
2
Colaborators

• Sem Lampotang (Anesthesiology)
• Cynthia Kaschub (Psychology)
• David Lizdas (Anesthesiology)
• And for jump-starting this effort
• Sue Legg (Director, Partnership for Global
  Learning)

3
Plan for the talk

• Learning with understanding The idea of
  mental models in psychology and education
• How multimedia presentations can boost learning
• Potential advantages of simulation
• Transparent simulations and understanding
• The Virtual Anesthesia Machine (VAM)
• Learning with Transparent versus Opaque VAM
• Bridging abstract and concrete models Mixed
  Reality and the Augmented Anesthesia Machine
• A little bit about individual differences

4
The mini-science of learning

• What makes a difference?
• Amount of practice (and the Power Law)
• Distribution of practice (and the Spacing Effect)
• Quality of practice (and Depth of Processing)
• Making the information distinctive
• Building appropriate mental models

Im doing great in all my other classes. I read
the book, came to class, outlined the material,
and made flash cards, and still got a C. Well,
did you understand the material? I thought I
did
5
Mental models and schemas in comprehension
If the balloons popped, the sound wouldnt be
able to carry since everything would be too far
away from the correct floor. A closed window
would also prevent the sound from carrying, since
most buildings tend to be well insulated. Since
the whole operation depends on the steady flow of
electricity, a break in the middle of the wire
would also cause problems. Of course, the fellow
could shout, but the human voice is not loud
enough for the sound to carry that far. An
additional problem is that the string could break
on the instrument. Then there would be no
accompaniment to the message. It is clear that
the best situation would involve less distance.
6
(No Transcript)
7
Mental models in cognitive science

• Term first used by Kenneth Craik (43)
• If the organism carries a small-scale model of
  external reality and of its own possible actions
  within its head, it is able to try out various
  alternatives, conclude which is the best of them,
  react to future situations before they arise,
  utilise the knowledge of past events in dealing
  with the present and future, and in every way to
  react in a much fuller, safer, and more competent
  manner to emergencies which face it. (Craik, The
  Nature of Explanation, 1943)
• Quality of the model depends on how well it
  captures the features of the domain that are
  critical for the task at hand

8
Understanding problems (Greeno, 1977)

• Our internal representation (or model) of the
  problem should have
• accurate CORRESPONDENCE between relevant elements
  in the world and model
• good COHERENCE between elements in the model
• appropriate links to PRIOR KNOWLEDGE that can aid
  problem solving

correspondence
links to prior knowledge
coherence
mental model of problem
environment
9
Little things (can) mean a lot(aka the devils
in the details)

• Subtle changes in problem framing can have
  drastic effects on performance
• Effects of analogy on solving the X-ray problem
• Preceded by bulb filament problem
• fragile glass framing 33 then solve X-ray
• laser intensity framing 69 then solve X-ray
• Effects of lives lost/saved on risky decisions
• Disease control programs, one more risky
• Lives saved framing 22 choose risky action
• Lives lost framing 75 choose risky action

10
Pictorial Representations

• Came before text, historically
• Illustrations and drawings
• To illuminate structure, function and relations
• Animations and videos
• To make system dynamics visible
• Interactive simulations
• To actively explore cause-effect dynamics, test
  hypotheses, etc.
• Advantages of multimedia can be dramatic

11
Mayers work on Multimedia (e.g., How
lightning forms)

• Compares..
• text-only to text-with-illustration (often
  schematic)
• Narration-only with narration-plus-animation
• Tests..
• Retention by free recall of presented facts
• Transfer (understanding?) by generating solutions
  to
• Redesign (how could you decrease lightning
  intensity?)
• Troubleshooting (how could there be clouds, but
  no lightning?)
• Prediction (what would happen with lower air
  temperature?)
• Abstraction of Principles (What causes
  lightning?)

12
Retention and transfer with MM

• Retention Modest MM gains
• Across 6 studies, 23 gain, 0.67 effect size
• Transfer Dramatic MM gains
• Across 6 studies, 89 gain, 1.50 effect size

13
Potential advantages ofComputer-based simulation

• Cost cheap systems, easy to replace, low risk
• Track performance and provide just-in-time
  feedback on performance
• Virtually Real when needed
• But Reality can be played with
• Increase likelihood of rare but important events
• Increase salience of important features
• Present hyper-real depictions of space and time
• Make the abstract concrete, and the invisible
  visible

14
Instructional Choice-Points

• What do we want them to learn?
• Declarative knowledge, procedural skills
• Immediate or long-term retention
• Reproductive or creative, flexible learning
• How do we structure the learning?
• Amount of grounding in the domain
• Balance of guided (reception) and free
  (discovery) learning
• Amount of online assessment and intelligent
  tutoring
• Student-tailored, or one-size-fits-all

15
Opaque versus Transparent Reality

• Opaque representation Simulation may be closely
  analogous to the physical system (iconic,
  concrete, high-fidelity, virtual reality) but
  hides underlying structure, functions and
  relations
• Transparent representation Simulation sacrifices
  physical fidelity but makes underlying aspects of
  system overt (abstract, idealized, schematic,
  symbolic)

16
Transparency in simulations

• Hollans STEAMER (1981)
• Goldstones Concreteness Fading (2004)
• Butchers simplified diagrams (2006)
• Debate focusses on extent of fidelity and
  whether detail helps or hurts
• Little direct comparisonof simulation formats

17
The Opaque-Reality VAM
18
The Transparent-Reality VAM
19
The Virtual Anesthesia Machine wide use, little
data

• 10 man-years of development time
• Available for free to individuals on the web
• Over 10,000 registered users
• Many positive reviews, both formal and informal
• Our goal assess the effectiveness of VAMs
  Transparent Reality approach to simulation

20
Training Session

• 30-page instructional guide developed
• Provides foundation of knowledge
• About anesthesia
• About the anesthesia machine and its subsystems
• Guided tour of several subsystems
• Breathing circuit
• Mechanical ventilation
• Manual ventilation
• Stresses visualization of dynamics using VAM

21
Workbook Sample text

• Question 1 elimination of CO2. Are the gases
  exhaled by a patient scrubbed of CO2 before
  entering the bellows during mechanical
  ventilation?
• Demonstration using VAM Simulation
• ____ Click Reset to start simulation afresh
• ____ Point to the O2 flowmeter control knob to
  enlarge it, then click-and-hold, and drag it
  counterclockwise until the O2 bobbin inside is
  about halfway up the tube.

22
Workbook sample text (contd)

• What does this do? What happens to the flow of O2
  from the supply line?
• Opening the valve increases the flow of O2 from
  the supply line into the breathing circuit.
• Trace along its route through the plumbing. Where
  does it wind up?
• It depends. For example, If mechanical
  ventilation is selected, but not on, the O2
  flows backward through the CO2 absorber, past
  the bellows and into the scavenger system

23
Performance on Day 2 Testsundergraduate health
majors
24
Performance on Day 2 Tests 2nd-year medical
students
25
Performance on AAA Board Exam Review questions
(4AFC)
26
Judgments about VAM

• Confidence Judgments
• On Component function
• Significantly higher for Transparent VAM (p lt
  .01)
• On System dynamics
• Marginally higher for Transparent VAM (p lt .15)
• Preferences for additional study
• 17 of 20 in Transparent group (UG) prefer TR VAM
• 11 of 20 in Opaque group prefer (UG) TR VAM
• 2 in TR, 7 in OR, think both would be preferable
  to either
• Similar trends among medical students more want
  both

27
Where to next?

• Combination and order effects
• Goldstones concreteness fading method?
• More precise tests of transfer
• Transfer to procedural skill does TR improve
  error detection and response?
• Hybrid simulations John Quarles project

28
The Augmented Anesthesia Machine (AAM)

• Integrating transparent and realistic
  representations with mixed-reality simulation

John Quarles and his Magic Lens
29
Declarative and Procedural Knowledge with VAM and
AAM

• Two groups of undergrads
• Training
• Introduction to AM with VAM
• positioning components within the actual AM
• Five step-through exercises with VAM or AAM
• Day 2 Testing
• Declarative Board Exam Questions
• Procedural Find a machine fault in the AM

30
Performance with VAM vs. AAM
31
Abstract and Concrete Knowledge

• Although the VAM may offer improved abstract
  knowledge, participants found it difficult to
  transfer this knowledge to the concrete
  anesthesia machine. This is precisely the concern
  that anesthesia educators have had with the VAM.
• For example, many VAM participants understood the
  abstract concept of the inhalation valve and they
  correctly answered the written questions
  regarding the gas flow in the valve. However,
  during the fault test, they could not perform the
  mental mapping between the abstract
  representation of the VAM inhalation valve and
  the concrete representation of the real
  anesthesia machine inhalation valve. Thus, it was
  difficult for VAM participants to apply their
  abstract knowledge to a concrete problem, such as
  the problem presented in the fault test.

32
Role of Spatial Abilities?

• Three tests of spatial cognition
• Arrow-pointing working memory (small-scale)
• Perspective-taking (mid-scale)
• Navigating virtual environment (large-scale)
• Correlations of spatial abilities and performance
  tend to be larger with VAM than AAM
• Suggests those with strong visualization skills
  can compensate for impoverished materials

33
What weve learned

• Dynamic simulations can improve comprehension of,
  and memory for, complex systems, BUT -
• Different kinds of simulation are optimum for
  different kinds of learning
• So we need to know the goal of training
• Experience with both abstract (schematic) and
  concrete (hi-fidelity) simulations may be optimum
• So we may need an integrated approach
• Individual differences in domain-specific skills
  and abilities will impact effectiveness of
  representations
• But we need to know how much

34
It takes a Village (or at least a Learning
Community)

• Cognitive/human factors psychologists
• Usability analysts
• Instructional psychologists and educators
• Simulation designers and engineers
• Domain experts and professionals

35
Thanks to all those RAs

• Emily McAlister
• Jonathan Greenwood
• Julianna Peters
• Shannon Bowie
• Sheila Holland
• Trudy Salmon