X Ray Vision in Augmented Reality

1
X Ray Visionin Augmented Reality

• Dieter Schmalstieg

2
Whats the big deal?

• We are not supposed to see through things!
• Brain rejects the idea of x-ray
• Accurate interior and exterior models (for
  registration)
• How to correctly Display X-Ray?
• How to give correct Depth information
• We must fool our brains

3
The Depth problem (mono)
4
Depth Static Cues

• Occlusion
• Relative height
• Relative size
• Atmospheric perspective (distance fog)
• Perspective distortion
• Brightness
• Texture Details
• Depth from Focus (stereo)

5
Depth Dynamic Cues

• Motion Parallax
• Obtained by
• Movement of the scene
• Movement of the camera (head)
• Speeds of two equivalent objects as a function of
  the distance
• Far distant objects appear to move slower than
  near objects
• Our eyes move constantly!
• But information is filtered by the brain

6
Depth Disparity

• The difference in distance of the projected
  position of the same point
• Used in stereo
• 3D Reconstruction

7
Depth Static Cues Example

• Texture Details
• Perspective
• Occlusion
• Relative Size

8
Display Medical AR Techniques
Opaque Surface Model
Transparent Surface Model
Volume Rendering window on skin
Wireframe
Volume Rendering window on skin
Glass skin
Setup
Volume Rendering
T. Sielhorst, C. Bichlmeier, S.M. Heining, N.
Navab, Depth perception a major issue in medical
AR Evaluation study by twenty surgeons, In
Proceedings of Medical Image Computing and
Computer-Assisted Intervention (MICCAI 2006),
Copenhagen, Denmark, October 2006
9
Display Context Sensitive Magic Lenses

• Interactive Tool for Context Sensitive
  Visualization
• Displays objects according to their contextual
  information
• Minimizes the number of used tools by exploiting
  semantic information
• Enables
• X-Ray
• Information Filtering
• Information Enhancement

Mendez, Kalkofen, Schmalstieg, ISMAR 06
10
Display X Ray by Layers

• Developed at UCSB
• Allows visualization by layers
• Helps with Superman X Ray Problem
• Allows selection of information by the user

Bane Hoellerer, ISMAR 04
11
Questions?