Medical Image Computing: From Data to Understanding

1
Medical Image Computing From Data to
Understanding

• Ron Kikinis, M.D.,
• Professor of Radiology, Harvard Medical School,
  Director, Surgical Planning Laboratory, Brigham
  and Womens Hospital

Founding Director, Surgical Planning Laboratory,
Brigham and Womens Hospital Principal
Investigator, the National Alliance for Medical
Image Computing, and the Neuroimage Analysis
Center Research Director, National Center for
Image Guided Therapy
2
Acknowledgments

• F. Jolesz, C. Tempany, P. Black, S. Wells, CF.
  Westin, M. Halle, S. Pieper, N. Hata, T. Kapur,
  A.Tannenbaum, M. Shenton, E. Grimson, P.Golland,
  W.Schroeder, and many more.

3
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

Science several hundred peer-reviewed scientific
papers since 1990
4
MIC The Problem

• More image data, more complexity
• Medical Image Computing aims to extract relevant
  information from images

5
MIC The Science

• Algorithm research
• Software tool development
• Biomedical research (applications)

Courtesy R. Jose et al.
Courtesy P. Black et al.
6
MIC The Approach

• Research and development conducted by
  interdisciplinary teams

Pohl et al.
7
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

8
The SPL

• A local resource with national impact
• Specialized in interdisciplinary research
• Network of strong collaborations

9
The SPL Organization

• Team of scientists (Wells, Westin, Halle, Hata,
  Talos, Shenton, Tempany)
• Software Engineering group (Pieper)
• Develops 3D Slicer and other applications
• Operations group (McKie)
• Maintains IT environment, including servers,
  storage, network, mail, web

Jose et al.
10
SPL IT Infrastructure

• IT environment designed to enable science
• Sized for easy access No scheduling
• Local Services
• Global Services
• Local Control

11
SPL
Courtesy M. Halle
12
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

13
EM Segmenter

• Evolution 1993-2007
• Segmentation based on statistical classification
• Models of
• Signal Intensities
• Noise
• Regions
• Variability

14
EM 1993 Signal Intensities

• Strengths
• Self-adaptive
• Corrects for gain fields
• Weaknesses
• Sensitive to noise
• No anatomical knowledge
• Adaptive Segmentation of MRI Data. WM Wells III,
  WEL Grimson, R Kikinis, FA Jolesz. IEEE
  TRANSACTIONS ON MEDICAL IMAGING, VOL. 15, NO. 4,
  AUGUST 1996

15
EM 1998

• Adding Mean Field Correction
• Models of Noise

Enhanced Spatial Priors for Segmentation of
Magnetic Resonance Imagery. T. Kapur, W.E.L.
Grimson, W. M. Wells III, R. Kikinis, MICCAI,
Cambridge, MA, Octobery 1998
16
EM 2004 Regions
Automated Parcellation K.M. Pohl, et al.
Anatomical Guided Segmentation with
Non-Stationary Tissue Class Distributions in an
Expectation-Maximization Framework, ISBI, pp.
81-84, 2004
17
EM 2006 Variability
MRI
LogOdds
uncertain
outside
inside
Log Odds an implicit shape representation
Pohl et al. Logarithm Odds Maps for Shape
Representation. Proceedings of the 9th
International Conference on Medical Image
Computing and Computer-Assisted Intervention
(MICCAI 2006), Copenhagen, Denmark, October 1-6,
2006, LNCS 4191, pp. 955-963.
18
EM 2007 Algorithm ? Tool
B. Davis, S. Barre, Y. Yuan, W. Schroeder, P.
Golland, K. Pohl
19
CT of the Hand

• EM Segmentation of the Phalanx Bones of the Hand
• Nicole Grosland, Ph.D.,1 Vincent A. Magnotta,
  Ph.D.,2 Austin J. Ramme3
• 1 University of Iowa Department of Biomedical
  Engineering, 2 University of Iowa Department of
  Radiology, 3 University of Iowa Carver
  College of Medicine

20
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

21
Diffusion Tensor Imaging
1997
Westin CF, Peled S, Gudbjartsson H, Kikinis R,
Jolesz FA. Geometrical diffusion measures for MRI
from tensor basis analysis. In ISMRM '97.
Vancouver Canada, 19971742.
22
2D Diffusion tensor display

1997-98
2D display for visualizing tensors (Peled et al
1998). Blue lines show the in-plain orientation
of the major diffusion direction. Out-of-plane
diffusion component color coded.
23
Tracking of WM Fibers
1999
green corpus callosum fiber disruption by GBM
red optic radiation green genuculocalcarine
tract light green auditory radiation
Provided by Meier, Mamata, Westin et al. 1999
24
DT-MRI Tractography
Provided by Westin, Mamata, et al, 2000
25
Fiber Clustering
2004

• Fiber bundle clustering using spectral methods
• Pair-wise fiber affinities are inserted in a
  large matrix
• Eigenvectors of this matrix define manifold

Provided by A. Brun
26
Visualization
2006

• Automatic extraction of anatomically meaningful
  fiber bundles.
• Advanced Rendering methods for segmentation
  results using photon mapping

Rendering provided by Banks, Data by Odonnell,
Shenton, Westin, et al., 2006
27
Validation
Courtesy L. Odonnell

• Validation is a keystone of the scientific method
• How do we validate these advanced mathematical
  concepts?
• Validation is an elusive goal in MIC

28
Validation using Histology
2005
In vivo DT-MRI Macaque monkey Craniotomies
were performed and 4 WGA-HRP was
pressure-injected into primary visual cortex
(V1), primary motor cortex Ex vivo DT-MRI of
fixed brain 4.7T using spin-echo DWI (30
directions, b-values of 1000 s/mm2), with voxel
dimensions 0.5x0.5x1mm3
CF Westin, LMI, SPL Sharon Peled HCNR, Harvard
Medical School Richard Born, Department of
Neurobiology, Harvard Medical School Vladimir
Berezovski, Department of Neurobiology, Harvard
Medical School
Using procedures approved by the Harvard Medical
Area Standing Committee on Animals
29
Setup
30
Coronal Sectioning with Cryostat
Digital imaging during sectioning was performed
in order to capture the undistorted brain in a
fixed coordinate system for subsequent 3D
reconstruction
- Sectioning at Neurobiolgy (V. Berezovski)? - 25
Mpixel Hasselblad camera (P. Ratiu)?
31
Validation HRP histology
Scanned 80 µ thick histological section showing
WGA-HRP-stained tracts. Five tracts can be seen
originating in the post-central gyrus
(somatosensory cortex).
3D Histological Reconstruction of Fiber Tracts
and Direct Comparison with Diffusion Tensor MRI
Tractography Julien Dauguet, Sharon Peled,
Vladimir Berezovskii, Thierry Delzescaux, Simon
K. Warfield, Richard Born, Carl-Fredrik Westin,
Ninth International Conference on Medical Image
Computing and Computer-Assisted Intervention
(MICCAI'06), Copenhagen, Denmark 2006
32
Validation Tractography
Top Histology Bottom DTI
Dauguet J, Peled S, Berezovskii V, Delzescaux T,
Warfield S, Born R, Westin C. Comparison of fiber
tracts derived from in-vivo DTI tractography with
3D histological neural tract tracer
reconstruction on a macaque brain. Neuroimage.
2007 Aug 1537(2)530-8.
33
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

34
HPC in IGT
Nikos Chrisochoides Center for Real-Time
Computing The College of William and Mary

• fast ( lt 5 min end-to-end time)?
• fault-tolerant (many CoWs)?
• easy-of-use with 3D Slicer over the Internet

() Supported in part by NSF grants CSI-0719292,
ITR-0426558, ACI-0312980 and John Simon
Guggenheim Foundation.
35
Current Implementation
CWM
BWH
Toward real-time image guided neurosurgery using
distributed and grid computing (with Andriy
Fedorov, Andriy Kot, Neculai Archip, Peter Black,
Olivier Clatz, Alexandra Golby, Ron Kikinis, and
Simon K. Warfield. In Proceedings of the 2006
ACM/IEEE Conference on Supercomputing, Tampa,
Florida, November 11- 17, 2006.
() Non-rigid alignment of preoperative MRI,
fMRI, DT-MRI, with intra-operative MRI for
enhanced visualization and navigation In
image-guided neurosurgery (with N. Archip, O.
Clatz, A. Fedorov, A. Kot, S. Whalen, D. Kacher,
F. Jolesz, A. Golby, P.Black, S. Warfield) in
NeuroImage, 35(2)609-624, 2007.
36
Integration with 3D Slicer
Interface to Slicer through plug-in module
Courtesy N. Chrisochoides
37
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

38
Slicer 3

• Next Generation
• At least 80 of code rewritten
• gt 500K lines of code
• Improved Look and Feel (KWWidgets)
• Improved Modularity
• Analysis routines can be used as plugins or
  command line executables for batch processing
• Draws on Multi-Institution Community

• Google slicer 101

38
Courtesy S. Pieper
39
Slicer Features

• Multi-Plattform
• Visualization
• Filtering
• Registration
• Segmentation
• DTI
• Quantification
• IGT Capabilities device interfaces
• Plug-in architecture
• Interfaces into informatics frameworks

• Specialties Involved
• Medical Imaging
• Applied Math
• Software Engineering
• Visualization
• Statistics
• Computer Vision
• Neuroscience
• Robotics
• User Interface
• Information Design

40
Informatics
Courtesy W. Plesniak

• Query Atlas
• XNAT

Courtesy S. Pieper
41
Beyond Medical
From Nanometers to Parsecs
Exploring Astrocytes Courtesy Brian Smith, Mark
Ellisman et al. National Center for Microscopic
Imaging Research
Detecting Outflows from Young Stars Courtesy of
Michelle Borkin, M. Halle, A Goodman et al.
Initiative in Innovative Computing, Harvard
42
Image Gallery
Many More Examples
42
43
Overview

• Introduction
• SPL
• Science
• EM
• DTI
• Engineering
• HPC
• Slicer
• Outlook

44
Beyond the SPL

• How do we make our science accessible beyond our
  collaborators?
• Dissemination using web, presentations,
  publications
• Sharing science and tools through
• Free Open Source Software (FOSS)
• Training and collaboration
• A community of developers and users
• Partnerships with Industry on our terms

45
NAMIC

• National Alliance for Medical Image Computing
• From local to wide-area
• One of seven National Centers for Biomedical
  Computing funded by NIH

Al Hakim et al.
46
NA-MIC An Alliance of Peers

• Leadership
• BWH Ron Kikinis, (Overall PI)?
• Core 1 Algorithms
• Utah Ross Whitaker (Core 1 PI), Guido Gerig?
• MIT Polina Golland, Eric Grimson
• UNC Martin Styner
• MGH Bruce Fischl, Dave Kennedy
• GaTech Allen Tannenbaum
• Core 2 Engineering
• Kitware Will Schroeder (Core 2 PI)?
• GE Jim Miller
• Isomics Steve Pieper
• UCSD Mark Ellisman, Jeff Grethe
• UCLA Art Toga
• Core 3 DBP 2004-2007
• BWH Martha Shenton
• Dartmouth Andy Saykin
• UCI Steve Potkin
• UofT Jim Kennedy

• Core 4 Service
• Kitware Will Schroeder
• Core 5 Training
• MGH Randy Gollub
• Core 6 Dissemination
• Isomics Steve Pieper, Tina Kapur
• Core 7 Management
• BWH S. Manandhar, R. Manandhar

Provided by Pieper, Kikinis
47
NA-MIC is Big Science

• Plus
• Big Science can be a force multiplier
• Development and adoption of best practices
• Faster and higher-quality dissemination of new
  techniques and of new science
• Minus
• Change in culture needed
• Replace
• My research
• with
• Our research

48
FOSS in NA-MIC

• Free
• Open Source
• No restrictions on use
• No requirement to give back derived code (you
  decide how much you want to share)?
• Software

I. Courouge et al.
49
The FOSS Value Proposition

• Cost effective Reduced duplication
• High quality Openness enables validation,
  debugging and local control
• Lowers barriers for scientific exchange

Fletcher et al.
50
The NA-MIC Kit

• Designed for Research (but compatible with
  commercial activities)
• FOSS 3D Slicer, ITK, VTK, KWW
• Software engineering methodology
• Portable multi-platform cmake
• Multi-site development nightly builds dart
• Quality assurance automated testing ctest

Fischl et al.
51
MIC Outlook More and More

• More Data
• Acceleration of data production
• Will drive the need for
• More Algorithms
• New classes of algorithms
• More Software
• An increasing number of tools will be needed
• More Application Packages
• Automated analysis as a method of data reduction

Levitt et al.
52
And What Else?

• Many of these concepts are expandable
• Beyond imaging
• NA-MIC kit software methodology adopted by the
  KDE community
• Beyond biomedical
• Astronomical Medicine

53
More Information

• SPL website
• http//www.spl.harvard.edu
• NA-MIC wiki
• http//wiki.na-mic.org
• 3D Slicer
• http//www.slicer.org

53