VisualizeR: Software Architecture for a VRBased Learning Environment

1
VisualizeR Software Architecture for a VR-Based
Learning Environment

• Margaret Murray MA
• University of California, San Diego
• School of Medicine
• Learning Resources Center
• La Jolla, California USA

2
Virtual Reality and the Medical Curriculum

• (Show 1994 video )
• by Helene Hoffman, PhD

3
Introduction

• VR-MM design described (94)
• Feasibility study video mock-up
• Development plans for lessons outlined (95)
• Match technical strategy to educational needs
• First working model created (96)
• VR-MM application realized (97)
• Multi-modal 3-D software architecture
  (VisualizeR)
• Lessons developed app refined (98-99)

University of California, San Diego - School of
Medicine
4
Anatomic VisualizeR Goals

• Teach spatial and clinical reasoning
• VR-MM paradigm achievable apropos
• Interact using fully VR-based lesson
• Intuitive manipulation of anatomic models
• Use of tools not available in real world
• Learning through discovery experience
• Enable use of multimedia curricular resources
• Add richness depth to experience
• Link structural didactic knowledge

University of California, San Diego - School of
Medicine
5
First Implementation
Show 1996 video clip

• Pros
• Highly realistic 3D anatomic models
• Rapid prototyping using
• Scenegraph approach
• Open Inventor API
• .iv file format
• Mouse/keyboard interface
• Cons
• Slow performance unsatisfactory passive user
  interface
• 3D anatomic models not real enough

University of California, San Diego - School of
Medicine
6
NLMs Visible Human Project

• Large Datasets generated from slices
• Visible Male 1mm
• Visible Female 0.3mm
• Polygonal and volumetric models can be derived
• Issues
• Segmentation
• Nomenclature
• Size
• Storage
• Bandwidth

University of California, San Diego - School of
Medicine
7
Visible Male A Quick Tour
Source Univ. of Maryland at College Park
Human-Computer Interaction Lab
University of California, San Diego - School of
Medicine
8
VisualizeR 3-D Anatomic Models

• NLMs Visible Male -gt Visible Productions -gt UCSD
• Visible Productions (CO) derives polygonal models
• UCSD medical illustrator sculpts and refines to
  meet faculty teaching requirements
• TECHNICAL ISSUE polygons for intuitive motion
• Machine capability
• Mono vs stereo
• Size / number of organs
• EDUCATIONAL ISSUE Accuracy for faculty use
• Reduced by global decimation
• Idiosyncratic anatomic elements

University of California, San Diego - School of
Medicine
9
Gouraud Shading Wire Frame
Before
After
University of California, San Diego - School of
Medicine
10
Interface / Interaction Options

• Three types of visual displays
• Monoscopic CRT
• Stereoscopic CRT
• Head Mounted Display
• Multiple input devices/combinations
• Mouse
• 3-D trackball
• Gloves with motion trackers
• Simple gestural syntax

University of California, San Diego - School of
Medicine
11
University of California, San Diego - School of
Medicine
12
Anatomic VisualizeR

• Tools
• 3-D Reference
• Scale
• Unlink/Link
• Transparency
• Reorient
• X-sections
• Flag
• Probe
• Highlighter
• Ruler
• Draw tool
• Study Guide
• Grid
• Search

Study Guide Organizes Resources Key
concepts Exercises Selectable Links
University of California, San Diego - School of
Medicine
13
VisualizeR Software Architecture

• Domain-Specific Application
• User Interface Manager
• Content Grouop Manager
• Lesson Manager
• Blocks
• Resource-specific or Application tools
• Dynamically loaded only when needed
• 3D Xvr Widgets for Interaction
• FACET Event Manager
• Load, Unload, Select, Manipulate
• Device Drivers

University of California, San Diego - School of
Medicine
14
User Interface Settings

• Virtual Environment (VE) settings
• Device settings
• Generalized Xresources mechanism
• Application defaults
• User-settings selectively override default value

University of California, San Diego - School of
Medicine
15
VR Content Editors

• 3D Models
• VPSculpt (Visible Productions)
• Amapi3D (TGS)
• 2D Resources
• Adobe Photoshop
• Any text editor
• Digital Media Tools (SGI)
• Others
• Loaders for Specific File Formats
• Developed as needed

University of California, San Diego - School of
Medicine
16
Lesson Editor
ENTRIES appear in the Study Guide

• GUI for faculty authoring
• Study Guide organizes entries and actions
• Specifies and places
• 3-D Models
• Images
• Video
• Lessons are ASCII files
• Modifiable by any text editor

ACTIONS take effect when a 3D button is
selected or when a page is turned.
University of California, San Diego - School of
Medicine
17
Device Interface Editor
AVATARS assigned. (3D model associated
with device)

• Core Commands
• Select
• Manipulate
• Remove
• Device-specific Commands
• Generates external ASCII command mapping file.
• Command mapping can be changed without
  recompiling the app.

Ascension Flock of Birds Spaceball 3003 Sensable
PHANTOM Fakespace Pinchgloves Immersion
Microscribe
University of California, San Diego - School of
Medicine
18
3D Perception Metrics Experiment

• Protocol
• Every combination run each day by each subject
• Conditions counterbalanced to avoid order effect
• Very simple 3D task
• NASA Task Load Index Questionnaire
• Kennedy Simulator Sickness Questionnaire
• Human Subjects Plan
• Concern over cybersickness
• 15 subjects (medical students)
• 4 sessions, 4 different days, 1 to 1.5 hrs each
  session
• Statistical Analysis and Results

University of California, San Diego - School of
Medicine
19
VR HW Configurations

• Non-immersive Monoscopic
• Partially immersive Stereoscopic
• Fully immersive HMD

University of California, San Diego - School of
Medicine
20
SW Layout Options
University of California, San Diego - School of
Medicine
21
Lesson Modules
Organize lesson resources Guide and
mentor Encourage active learning
22
VisualizeR in Secondary Education

• High school advanced anatomy course
• Lessons primarily authored by teacher
• Part of faculty-development activities
• Adjunctive learning experiences
• Introduction 5 lesson modules
• Problem-based approach
• Measure length of spinal column
• Construct skull and spine from parts
• Visualize deep abdominal structures
• Used during visits to School of Medicine

University of California, San Diego - School of
Medicine
23

VisualizeR in Medical Education

• First VR at UCSD School of Medicine
• Incorporated into Human Anatomy course
• Lesson developed by UCSD faculty
• Focus sphenoid cranial nerves
• Used by 2nd year medical students
• Implemented in two contexts
• Interactive demonstration in lecture
• Monoscopic version
• Student use in VR laboratory
• Stereoscopic version

University of California, San Diego - School of
Medicine
24
Anatomic VisualizeR Today

• (Show 1999 video )

25

Future Plans

• Migrate VisualizeR from research lab to 21st
  century classroom
• Continue lesson development
• Develop low cost client version (NT or Linux)
• Extend understanding of learning in VEs
• Educational outcomes
• Cost/benefit
• Explore alternative funding options
• Educational grants
• Commercial partnerships

University of California, San Diego - School of
Medicine
26
Conclusions

• VR-MM approach enriches anatomy education
• VisualizeR architecture can be applied to any
  curriculum requiring knowledge of spatial
  relationships
• Physics
• Biochemistry
• Architecture
• Current research culminated in pilot projects
• High school (advanced anatomy students)
• Medical education (med students, nursing
  students)
• Complete curriculum still to be developed

University of California, San Diego - School of
Medicine
27
Acknowledgements

• Grant funding
• Defense Advanced Research Projects Agency
• Office of Naval Research
• Software development
• Robert Curlee
• Mark Danks
• Ram Prayaga
• 3D model sculpting
• Alicia Fritchle
• Ongoing vision support
• Dr. Richard Satava

University of California, San Diego - School of
Medicine
28
For more information

• Margaret Murray, MA or Helene Miller
  Hoffman, PhD (P.I.)
• Learning Resources Center
• University of California, San Diego
• School of Medicine
• La Jolla, California, USA 92093-0661
• (858)-534-4123
• mmurray_at_ucsd.edu hhoffman_at_ucsd.edu
• http//cybermed.ucsd.edu