CSci 6971: Image Registration Lecture 16: View-Based Registration March 16, 2004

1
CSci 6971 Image Registration Lecture 16
View-Based RegistrationMarch 16, 2004
Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware
2
Overview

• Retinal image registration
• The Dual-Bootstrap ICP algorithm
• Covariance matrix
• Covariance propagation
• Model selection
• View-based registration
• Software design
• Warning mathematically, this lecture is a
  little rich. You will not be responsible for
  knowing the details

3
Retinal Image Registration Applications

• Mosaics
• Multimodal integration
• Blood flow animation
• Change detection

4
Mosaics
5
Multimodal Integration
6
Fluorescein Angiogram Animation
7
Change Visualization
8
Retinal Image Registration - Preliminaries

• Features
• Transformation models
• Initialization

9
Features

• Vascular centerline points
• Discrete locations along the vessel contours
• Described in terms of pixel locations,
  orientations, and widths
• Vascular landmarks
• Pixel locations, orientations and width of
  vessels that meet to form landmarks

landmarks
vascular centerlines
10
Transformation Models
Model Parameter Matrix DoF Accuracy (pixels)
Similarity 4 5.05
Affine 6 4.58
Reduced quadratic 6 2.41
Full quadratic 12 0.64
11
Initializing Registration

• Form list of landmarks in each image
• Form matches of one landmark from each image
• The selection of these matches will be discussed
  in Lectures 18 and 19
• Choose matches, one at a time
• For each match
• Compute an initial similarity transformation in
  the small image region surrounding the landmarks
• Apply Dual-Bootstrap ICP procedure to see if the
  initial alignment can be successfully grown into
  an accurate, image-wide alignment
• End when one match leads to success, or all
  matches are exhausted

12
Dual-Bootstrap - Overview
Iterate until convergence

• Match and refine estimate in each region
• Bootstrap the model
• Low-order for small regions
• High-order for large
• Automatic selection
• Bootstrap the region
• Covariance propagation gives uncertainty

13
Matching and Estimation in Each Region

• Matching - standard stuff
• Vascular centerline points from within current
  region of moving image
• Mapped using current transform estimate
• Find closest point using Borgefors digital
  distance map
• Estimation
• Fix scale estimate
• Run IRLS

14
Covariance Matrix of Estimate

• Measures uncertainty in estimate of
  transformation parameters
• Basis for region growth and model selection
• The next few slides will give an overview of
  computing an approximate covariance matrix
• Well start with linear regression

15
Problem Formulation in Linear Regression

• Independent (non-random) variable values
• Dependent (random) variable values
• Linear relationship based on k1 dimensional
  parameter vector a

16
Least-Squares Formulation

• Least-squares error term
• Here

17
Estimate and Covariance Matrix

• Estimate
• Residual error variance (square of scale)
• Parameter estimate covariance

18
Aside Line Fitting in 2D

• Form of the equation
• If the xi values are centered
• Then the parameters are independent with
  variances
• for the linear and constant terms, respectively

19
Hessians and Covariances

• Back to k dimensions, re-consider the objective
  function
• Compute the Hessian matrix
• Observe the relationship

20
Hessians and Covariances

• This is exact for linear regression, but serves
  as a good approximation for non-linear
  least-squares
• In general, the Hessian will depend on the
  estimate (in regression it doesnt because the
  problem is quadratic), so the approximate
  relationship is

21
Hessian in Registration

• Recall the weighted least-squares objective
  function
• Keeping the correspondences and the weights
  fixed,
• where Dk gives the error of the k-th
  correspondence
• Inverting this gives the covariance
  approximation.
• This approximation is only good when the estimate
  is fairly accurate

22
Back to Dual-Bootstrap ICP

• Covariance is used in two ways in each DB-ICP
  iteration
• Determining the region incorporates enough
  constraints to switch to a more complex model
• Similarity gt Affine gt Reduced Quadratic gt
  Quadratic
• Determining the growth of the dual-bootstrap
  region
• More stable transformation estimates lead to
  faster growth

23
Model Selection

• What model should be used to describe a given set
  of data?
• Classic problem in statistics, and many methods
  have been proposed
• Most trade-off the fitting accuracy of
  higher-order models with the stability (or lower
  complexity) of lower-order models

24
Model Selection in DB-ICP

• Use correspondence set
• Estimate the IRLS parameters and covariance
  matrices for each model in current set
• For each model (with dm parameters) this
  generates a set of weights and errors and a
  covariance matrix
• Choose the model maximizing the model selection
  equation (derived from Bayesian modeling)
• The first two terms increase with increasingly
  complex models the last term decreases

25
Region Growth in DB-ICP

• Grow each side independently
• Grow is inversely proportional to uncertainty in
  mapping of boundary point on the center of each
  side
• New rectangular region found from the new
  positions of each of the boundary points

26
Aside Covariance Propagation and Transfer Error

• Given mapping function
• We will treat Q as a random variable, but not gk
• Uncertainty in Q makes gk a random variable.
• What then is the covariance of gk?
• We solve this using standard covariance
  propagation techniques
• Compute the Jacobian of the transformation,
  evaluated at gk
• Pre- and post-multiply to obtain the covariance
  of gk
• In computer vision, this is called the transfer
  error

27
Outward Growth of a Side

• Let hk be the outward normal of the side, and let
  rk be the distance of the side from the center of
  the region
• Project the transfer error covariance onto hk to
  obtain a scalar variance sk
• The outward growth (along normal hk) is
• where b controls the maximum growth rate, which
  occurs when sk lt 1

28
Putting It All Together - The Example, Revisited
29
Turning to the Software

• A view is a definition or snapshot of the
  registration problem.
• A view contains
• An image region (current region, plus goal
  region)
• A current transformation estimate and estimator
• A current stage (resolution) of registration
• Views work in conjunction with multistage
  registration

30
View-Based Registration - Procedural

• The following is repeated for each initial
  estimate
• For each stage
• Do
• Match
• Compute weights
• Estimate scale
• For each model
• Run IRLS to estimate parameters and covariances
• Re-estimate scale
• Generate next view
• For DB-ICP view generator this choose the best
  model and grows the region
• Until region has converged and highest order
  model used
• Prepare for next stage

31
Implementation

• rgrl_view
• Store the information about the view
• rgrl_view_generator
• Generate the next view
• rgrl_view_based_registration
• Mirrors rgrl_feature_based_registration with
  modifications based on the outline on previous
  slide
• Example
• rgrl/example/registration_retina.cxx

32
Summary

• Retina registration
• Models, features and initialization
• DB-ICP
• Matching, estimation and covariances
• Model selection
• Region growing
• Generalization to view-based registration and its
  implementation in the toolkit.

33
Looking Ahead to Lecture 17

• Discussion of toolkits
• Whats easy, whats hard
• Whats missing
• Project discussion
• Requirements
• Topics and initial steps