Presentation on Different Tissue Modeling and Surgical Simulation Groups

1

• Presentation on Different Tissue Modeling and
  Surgical Simulation Groups

2
Laparoscopic Visualization Group of Univ of NC,
Chapel Hill

• Augmented reality
• Depth extraction from laparoscopic images are
  used
• Head-mounted display with video-see-through
• Creates a three-dimensional visualization system

3
Contd

• camera head has the full six degrees-of-freedom
• real and synthetic images are merged together
• head mount enables motion parallax cues 3D
  effect normally available in open surgery.

4
The Visualization Laboratory at SUNY Stony Brook

• "3D virtual colonoscopy" using efficient volume
  rendering
• interactive navigation
• diagnose patients
• Spiral CT scan of the patient's abdomen used as
  input
• A virtual 3D colon is computed and displayed

5
Contd

• Volume Rendering
• Replaces more traditional surface rendering
• SR uses 3D segmentation (eg. Marching Cubes)
• Time-consuming (technically offline, but in
  practice online ie. clinically)
• Susceptible to false surface creation (if organs
  are not specifically prepared for procedure)
• Surface-rendered virtual colonic surface looks
  artificial compared to standard optical
  colonoscopy.

6
Contd

• A new, fast, ray-casting algorithm
• Called "potentialfieldassisted ray casting
• Achieves speedup without loss of image fidelity
• No intermediate surfaces need to be generated
  from the 3D data
• Instead a "transfer function" aids in displaying
  the volumetric data
• Controls levels of color and opacities
• A smoother, softer colonic surface image results
• More realistic (according to physicians)

7
Penn State University's Multidimensional Image
Processing Lab

• Quicksee Project
• Input high-resolution 3D radiological scans
• Uses offline 3D segmentation to create a
  navigation environment.

8
Contd

• Also, "Orthographic Projections Display presents
  following slices
• transverse,
• sagittal
• coronal
• "Mercator Reference Projection
• All directions at given point displayed

9
Contd

• A "Measurement Tool" provides graphical and
  numerical information
• cross-sectional area variation
• branching anatomical structures, such as
  pulmonary airways and coronary arteries
• branch lengths
• branch angles
• Allows interactive exploration along arbitrary
  paths

10
Contd

• Rendered structures use patient-specific image
  data, combined with model
• Uses a fast volume-rendering algorithm Real-time
  navigation
• (cf. SUNY at SB)
• Based on coherence-based ray-casting

11
Contd

• Quicksee
• allows unrestricted exploration of virtual
  anatomy
• based on patient-specific 3D radiological scans
• Serves as
• diagnostic tool
• surgical-planning environment.

12
Simulation Group at CIMIT

• Center for the Integration of Medicine and
  Innovative Technology
• Mark Ottensmeyer
• Organ deformation
• Measurement of properties in
• Healthy tissue
• Diseased tissue
• In vivo is preferable to in vitro
• Mathematical modeling

13
Measurement (stress-strain responses)

• Minimally invasive instrument
• acquires force-displacement response
• Liver
• Spleen
• Kidney
• Using normal indentation
• Or non-invasive imaging-based method

14
Non-invasive imaging method Elastography

• oscillatory displacements are applied to exterior
  surface of organ (ie. boundary condition)
• compression or
• shear
• Interior strain-field is measured using
• MRI or
• ultrasound.
• Elastic properties are thus determined

15
Limitations of non-invasive imaging method

• Because vibrations are small in amplitude, only
  linear portion of the (non-linear) response is
  measured
• Wavelength ltlt organ size, but the surgeon's
  typical hand movement generates waves of large
  wavelength (ie. low frequency)
• But response parameters are frequency dependent
  -gt experiments dont fully capture surgical
  frequencies

16
Limitations of force-displacement method

• Viscous nature of the response is not being
  measured
• dependent on strain-rate
• By contrast, using oscillations, both viscous and
  inertial material properties can be captured
• A frequency-dependent version of the modulus can
  be determined

17
Using force-displacement instrument

• A small indentation with a probe is made
• Using a mathematical expression that relates
• the curvature of the probe's tip
• the measured stiffness
• a geometry factor (eg. organ can be considered
  as semi-infinite for small probes and
  indentations)
• Youngs modulus
• The latter can then be computed

18
National Library of Medicine Visible Human 2

• Based on a denser acquired dataset
• Some of the previous artifacts have thus been
  removed
• Enhanced detection of structures is now possible
• Uses CT and MR scans, and axial cryosectioning
  (at 147 micron intervals and scanned at 2
  microns/pixel resolution)
• The domains of gross anatomy and histology have
  been bridged for the first time (resolution
  range 1 mm/voxel - 2 microns/pixel)

19
New Modeling Language

• Developed by Inria people and CIMIT people
• Called CAML "Common Anatomy Modeling Language
• Used for the development of next-generation
  surgical simulators
• Aims at removing communication seams between
  the Engineering, CS and Medical communities

20
Inria

• People Cotin, Delinguette, Ayache
• Soft Tissue Modeling
• 3 different linear elastic FEM models
• First method
• Offline evaluation of deformations allow
  real-time superposition of pre-computed solutions
• Topology changes, such as cutting of tissue, is
  not allowed

21
Contd

• Second method
• "tensor-mass" model
• A limited number of nodes are permitted to change
  their connectivity
• Allows real-time simulation of restricted cutting
• Third method hybrid of other two

22
Basic Concepts in Tensor Analysis and Linear
Elasticity

• (describe on blackboard)

23
Non-Linear Modeling

• A non-linear FEM method
• Allows large deformations to be simulated
• Uses the original Green-St Venant strain tensor
  (instead of linearized version)
• Additionally, incompressibility of organs is
  modeled by imposing an extra constant-volume
  constraint

24
Incompressibility constraint

• Ideally, this is not necessary since Poisson
  ratio can be set to ½
• However, recalling that two equivalent sets of
  parameters are
• Youngs modulus and Poisson ratio
• For a uni-axial stress axial and radial strains
• Bulk modulus and shear modulus
• Strain, given a shape-preserving and
  volume-preserving stress, respectively

25
Contd

• 3 moduli add like springs (or capacitors) in
  series
• Incompressibility can be modeled by making bulk
  modulus infinite (and shear modulus
  normal-valued)
• Numerical problem arises from this infinite bulk
  modulus value time-step in PDE must be made
  arbitrarily small!

26
Contd

• The external constraint approach allows correct
  modeling without the bulk-modulus/time-step
  problem
• Realistic deformations are thus simulated in
  real-time

27
MIT Artificial Intelligence Lab

• "3D Slicer Software
• Integrates several elements of image-guided
  medicine into a single portable and extendable
  environment.
• automatic registration
• semiautomatic segmentation
• 3D surface model generation
• 3D visualization

28
Contd

• Used intra-operationally
• Real-time scans (combined with pre-operative
  slices) update 3D view
• Surgical instruments are tracked to get the
  location of 2D slices used in update

29
Surface generation

• 3D segmentation used to create label maps,
  indicating type of tissue
• Bounding surfaces are then obtained using
  Marching Cubes
• Surface is represented as a collection of
  triangles
• Decimation technique reduces the number of
  triangles with little loss of rendering accuracy

30
Contd

• 3D Slicer
• Runs on top of OpenGL
• Uses Visualization Toolkit (VTK) for efficient
  processing
• VTK uses C objects to make a data-processing
  pipeline
• Helps with interactivity by storing output at
  each stage of pipeline in memory
• This allows change to be executed at the minimal
  depth within the pipeline (after an update from
  user-interface controls)

31
Modeling Language

• In addition a "Medical Reality Modeling Language"
  (MRML) has been developed
• Different datasets (eg. Different organs, each in
  its own reference frame) are integrated into one
  3D model
• A tree-like graph (similar to VRML or Java3D) is
  used
• Concatenates coordinate transforms, yielding a
  composite transform applied to leaf node

32
Contd

• Additionally, a 3D Virtual Endoscopy simulator
  has been built on top of 3D Slicer
• Used for diagnosis and surgical planning
• Allows interactive exploration of
  patient-specific 3D anatomical models

33
Registration Methods from MIT AI Lab

• Method to perform "multi-modal" registration
• Matching two images obtained by different
  modalities eg. MRI, CT, optical, etc.
• Here, data from MRI (magnetic resonance imaging)
  is matched to data from CT (computer tomagraphy)
  or from PET (positron-emission tomography)
• Concepts from Claude Shannons Theory of
  Information are used mutual information

34
Contd

• Optimal relative orientation of two volumetric
  images is found
• mutual information between the two is maximized
• Higher-level preprocessing (eg. segmentation) is
  unnecessary
• Powerful method!
• Method of correlation (min SSD) cannot be used
  here, since modalities are different
• Good addition to bag-of-tricks!

35
Another system from MIT AI Lab

• "AnatomyBrowser
• Rendering of 3D surfaces is separated into two
  steps for efficiency
• Traditionally visualization of 3D surface models
  requires (for interactive rates)
• graphics acceleration hardware
• computationally intensive software

36
Contd

• Method here strikes a compromise between
• dynamic scene rendering and
• the use of static images
• First step pre-rendering
• A set of surface models are rendered individually
  using high-end graphics hardware
• Intensity and depth maps are generated and stored
  in an intermediate format called a "multi-layer
  image"

37
Contd

• In the second step
• Final image is displayed using low-end hardware
  and Java software
• At this point, certain parameters can be
  dynamically manipulated
• number of displayed models
• position of the camera
• surface transparency and color
• Computational requirements are small

38
Contd

• Applications of this system include
• surgery planning
• segmentation assistance
• interactive anatomy atlases
• Used for model-driven segmentation
• Atlas acts as a deformable template
• An input grayscale 3D data set is registered to
  it
• Segmentation is performed by mapping from the
  atlas back to the input data