Visualization in the DARPA Virtual Soldier Project

1
Visualization in the DARPAVirtual Soldier Project

• Stewart Dickson, Richard Ward
• Oak Ridge National Laboratory
• This work was supported by a grant from the
  DARPA, executed by the U.S. Army Medical Research
  and Materiel Command/TATRC Cooperative Agreement,
  Contract W81XWH-04-2-0012.

2
Collaborators

• University of Michigan/Center for Biological
  Information
• University of California, San Diego
• University of Washington
• Crowley-Davis Research
• University of South Carolina
• University of Utah
• Oak Ridge National Laboratory

3
Virtual Soldier Phase I Demo

• P-Tag, CODEC, Holomer Display HotBox
  (Holographic Stereo Display)
• 2. Statistical Reasoning
• Backward from Effect to Cause
• 3. Multi-Scale Modeling
• Physiologic Predictive Model
• 4. Causal Reasoning
• Forward from Cause to Effect

4
Part 1

• P-Tag
• 2 Gigabytes Baseline Medical Data
• CODEC
• Automatic CT Volume Segmentation
• Holomer Display
• HotBox
• Interface to Virtual Soldier Knowledge Base
• (Holographic Stereo Display)

5
P-Tag, CODEC Data Flow -- CDR
6
Imaging and Segmentation -- GE
7
Imaging and Segmentation -- GE
8
Imaging and Segmentation -- GE
9
Imaging and Segmentation -- GE
10
Virtual Soldier Knowledge Base
11
Holomer Display -- Utah/ORNL
12
Holomer Display -- Utah/ORNL
3-D Cursor
13
HotBox -- ORNL
14
HotBox -- ORNL
15
HotBox Data Sources

• Anatomical Labels from Segmentation
• Adjacency Relations from Segmentation
• Entity Bounding Boxes from Segmentation
• Injury List from Causal Reasoning
• Surface Geometry from Segmentation
• Physiological Data from HIP Simulation
• Foundational Model of Anatomy

16
HotBox User Feedback

• Highlights Selected Tissue in 3-D Anatomy
• Updates Adjacent Tissues to Selection
• Updates Structural Relationships to Selection
• Alerts to Injured Tissue
• Visually Indicates Probability of Injury
• Dynamically Displays Physiological Data
• Associated with Selection

17
Part 2 Statistical Reasoning

• Backward from Effect to Cause Using
• Automatic CT Scan Segmentation and Labeling
• Morphometric Markup
• High-Integration Parameter Simulation (HIP)
• Virtual Soldier Knowledge Base (VSKB)
• Forecast Survival or Time to Death
• Estimate Injury Location Severity

18
Statistical Reasoning Data Flows
19
Decision Space Display -- U. Michigan
20
Probable Injury Display
21
Part 3 Multi-Scale Modeling

• Anatomical and Physiologic Predictive Model
• Circulatory physiology combined with
• Anatomically and Biophysically detailed
• 3D Models of Ventricular Electromechanics
• Model physiological signals not measurable in the
  field
• Database covers more cases than possible with
  experimental data alone

22
Multi-Scale Modeling Data Flows
23
Material Point Method -- Utah/UCSD
24
MPM Cardiac Penetration Trauma
25
EdgeWarp -- University of Michigan
26
EdgeWarp -- University of Michigan
27
Part 4 Causal Reasoning

• Help the Medic make Triage Decisions
• Given two anatomically similar wounds
• Reason from Cause to Effect
• Deduce Differences in Pathological
• Consequences
• Simulate Differences in Pathophysiological
• Outcomes

28
Causal Reasoning Data Flows
29
Simulated Cardiac Wound - Strain
30
Simulated Cardiac Wound - Pressure
31
Highly Integrated Physiology (HIP)
Models Available at http//nsr.bioeng.washington.
edu
32
Physiology Data Display -- U.Mich.
33
Causal Reasoning -- U. Washington
34
Foundational Model of Anatomy
35
Coronary Circulation Simulation
36
Bullet-Proof Heart -- UCSD/Utah