ECSE6963, BMED 6961 Cell

1
ECSE-6963, BMED 6961Cell Tissue Image Analysis

• Lecture 22 Tube Segmentation (contd)
• Badri Roysam
• Rensselaer Polytechnic Institute,
• Troy, New York 12180.

Center for Sub-Surface Imaging Sensing
2
Recap

• Two basic methods
• Extract the foreground and skeletonize by erosion
• Trace along the structures
• Vectorization/Tracing Algorithms
• Based on the idea that tubes exhibit parallel
  edges.
• Steps
• Survey the image via a sparse grid analysis
  find seeds
• Estimate image contrast from seeds
• Trace the structures starting from seeds
• Stopping criterion, and response threshold T
• Special cases to worry about branch points and
  parallel tubes
• Tradeoff accuracy, computation, and detection
  performance using the grid density and threshold

3
Recall Edge Detection with Pre-Smoothing
Approximate Gaussian Smoothing Filter
Differentiator
Single operation that combines both
(scale factor)
4
Smoothing Along an Edge
Compute the average response along an edge
Note If image brightness goes up from left to
right, the result response is positive ? The
sign of the response can be put to good use
5
Core Tracing Algorithm
M/2

• Applied template with end starting at pk
• Searched range d in 0..M/2
• /- 1 direction

d
d in interval 0..M/2 M max expected blood
vessel width 26
M/2
d
6
A Delicate Issue

• The exact location of a branch point is hard to
  find
• Why?
• Because the parallel edges model is violated
  near that location, so the traces can be rather
  inaccurate
• This issue is important if the locations of the
  landmarks is important as in image registration

7
Simple-minded idea
Too Far!
short
o.k.

• Look for other traces in a small neighborhood of
  a stopped trace and join with a straight line
• Not very reliable!

8
More Examples
9
What can we do About It?

• Define an exclusion zone around the detected
  intersection
• parallel edges model cannot be trusted here
• Fit local lines to the trace points outside the
  exclusion zone, and find their least squares
  intersection
• This is at least repeatable

10
Example
11
More Examples
Better!
Better!
Before
After
12
Repeatability Examples
Naïve Detection
Detection with exclusion zone idea
13
How to Trace in 3-D

• Assume that we have a volumetric image
• Things to do
• Need to extend our models to 3-D
• Need to be mindful of computational needs
• Make approximations when safe

14
3-D Tracing Model
Top
Left

• Generalized cylinder model.
• Over a short distance, dendrites and axons in the
  image field are well approximated by generalized
  cylinders with slowly changing diameter.
• To fit an ellipse, we ideally need six sets of
  points
• Approximate method
• Use four sets of templates, right, left, top, and
  bottom.

Right
Bottom
15
Lots of Templates

• In 3-D space, we need to worry about two angles ?
  and ? instead of one
• We have four borders to track (top, bottom, left,
  right)
• If each angle is discretized to N values, we are
  left with 4?N2 templates

16
The Length of Templates

• The number of discrete angles imposes a lower
  bound on template length
• At equality, templates along adjacent directions
  differ by at most one pixel at their far end
• On the other hand, the maximum length of
  templates is determined by the straightness of
  the dendrites
• Question How do we do this automatically?

17
3-D Templates Summary

• A template is defined in terms of
• Its type right, left, top, bottom
• Its length K
• Its direction
• Its shift direction (next page)

18
Efficient Seed Point Selection

• Basic Idea
• Project the 3D image onto the xy-plane to reduce
  computation.
• For structures that are bright against a dark
  background, use a maximum intensity projection
• Use minimum intensity projection, or invert image
  if structures are dark against bright background

19
Iterative Tracing Procedure - I
Start
Get Seed Point
Yes
Refine and
No
Stop ?
Estimate and
Set
20
Illustration
21
Estimation of Boundary Points
Given a point and a direction , the
corresponding boundary points are estimated using
a shift and correlate procedure, exactly as in
the 2-D case.
Similar strategy for the top, bottom, and left
templates
22
Minimizing the Effort

• Search in the neighborhood of the current
  direction.
• Works well if the diameter of the generalized
  cylinder is changing slowly, and tortuosity is
  low

Limit search to here
Dont bother here
23
Adaptively Setting Template Length

• Observe
• Template too short ? too little averaging
• Template too long ? cant keep up with curving
  vessels
• Tradeoff
• Maximize length-normalized template response!!

24
Re-centering Step

• Better estimates for and are computed
  according to

The same core idea as in the 2-D case. The
equations look more complicated in 3-D.
25
Setting the Step Size Automatically

• The adaptively estimated template lengths provide
  important hints
• Update equations

26
Better Stopping Criteria

• With images of fluorescently labeled structures,
  there is opportunity for localized fading
• A conservative stopping criterion will simply
  result in too much fragmentation
• More forgiving criteria are needed
• Basic Idea Learn from games
• M strikes and youre out

27
Stopping Criteria

• Rationale.
• Tolerate responses that are characteristic of the
  background as long as such responses are isolated
  events.
• Conditions.
• Criteria.
• Stop the current tracing cycle if the number of
  consecutive violations is larger than a threshold
  ? (say 3).

C contrast F - B
At least one gray level of contrast
Response at 90o
28
Segmentation of Soma

• Basic Idea
• First find the soma in the 2-D projections
  follow up with a 3-D segmentation over a limited
  region
• Grayscale closing, adaptive thresholding, and
  connected component analysis.
• Grayscale closing.
• Erosion.
• Dilation.
• Thresholding.

29
Sample 3D image
XY
YZ
Neuron TR053Z1A Step Size Zoom
1.0 Dimensions 512x480x244x8
XZ
30
XY
YZ
Projections of the resulting traces (Takes about
a minute on a Intel Pentium III)
XZ
31
Dealing with Noise
32
Dealing with Noise

• The correlation kernels (templates) are based on
  assuming a Gaussian step edge, and little noise
• With noisy images, algorithm can break down
• Things we can do
• Design custom templates to suit the edge and
  noise models
• Use the more forgiving stopping criteria
• Design the stopping criteria using the noise
  models explicitly

33
Example
This type of noise can be handled using a median
instead of a mean
F Foreground intensity B Background
intensity C Contrast F B N Noisy pixels
intensity value
34
Breakdown Point

• Robustness measure
• Breakdown point is defined as the minimum
  fraction of outliers that can cause an estimate
  to diverge arbitrarily far from the true
  estimate.
• Basic Idea
• Use median template response instead

35
Median vs. Average Response
Average Median
36
Tracing using median response
Tracing using average response
37
3-D Tumor Vessel Example
Day 4
Day 1
Day 2
Day 3
38
Average Response
Median Response
39
Beyond
40
Neuronal Spines
Collaboration Joshua Trachtenberg (UCLA)
time
41
Summary

• We have studied the basics of tube tracing
  algorithms
• This is an active field with new ideas being
  published every year
• Plenty of room for more ideas!
• Next
• Time-lapse microscopy
• Analyzing changes in microscope images
• Putting it all together!

42
Instructor Contact Information

• Badri Roysam
• Professor of Electrical, Computer, Systems
  Engineering
• Office JEC 7010
• Rensselaer Polytechnic Institute
• 110, 8th Street, Troy, New York 12180
• Phone [phone]
• Fax [phone]
• Email roysam_at_ecse.rpi.edu
• Website http//www.ecse.rpi.edu/roysam
• Course website http//www.ecse.rpi.edu/roysam/CT
  IA
• Secretary Laraine Michaelides, JEC 7012, (518)
  276 8525, michal_at_.rpi.edu
• Grader Nicolas Roussell (roussn_at_rpi.edu, Office
  JEC 6308, 518-276-8207)

Center for Sub-Surface Imaging Sensing