Martin J. Moene ? E.H. van Tol-Homan ? P.V. Ruijgrok

1
Image Processing for Video-rate Scanning Probe
Microscopy
I A f M 2 0 0 6

• Martin J. Moene ? E.H. van Tol-Homan ? P.V.
  Ruijgrok
• T.H. Oosterkamp ? J.W.M. Frenken ? M.J. Rost
• Kamerlingh Onnes Laboratory

2
Image Processing forVideo-rate Scanning Probe
Microscopy
Martin Moene ? Interface Physics ? Leiden
University ? The Netherlands
50 x 49 nm 300 K Au(110)
graphic by Prof.Dr. Richard Berndt, Kiel
University
3
Scanning Probe Microscopy

• 1981 Scanning Tunneling Microscope (STM) 1
• 1986 Atomic Force Microscope (AFM)
• Other variants

20 x 13 nm 300 K Si(111)
1 G. Binnig, H. Rohrer, C. Gerber, and E.
Weibel, Phys. Rev. Lett. 49, 57 (1982).
4
40s per Image1024 x 1024
90 x 90 nm Si(111)
5
27 IMAGES per second (64 x 64 pixels2)
Pan
Rotate
r e a l t i m e
Zoom
27 Hz
80 Hz
HOPG
Au(110)
2 M.J. Rost, L. Crama, P. Schakel, E. van Tol
et al. Rev. Sci. Instrum. 76 (2005) 053710
6
LeidenProbeMicroscopy.com
Feedback Drivers Scan Generator ADCs
STM Head
7
Stabilizing and Comparing Images

• thermal drift

50 x 49 nm 300 K Au(110)
8
Apply Image Stabilisation to

• Stay Focused
• Enable Quantitative Analysis (comparing images)

1st Solution Normalized Cross-correlation (NCC)

• A tool for both
• Image Stabilisation and
• Quantitative Analysis

9
What is Cross-correlation (CC) ?

• Simplified nano wire orsingle-atom row

x
10
What is Cross-correlation (CC) ?

• Simplified crystal surface

x
11
What is Cross-correlation (CC) ?
x
CC depends on offset and amplitude
12
Better Correlate Signal Form
13
Symmetric Computation

• CC(c) N-1?x0 f(c x - N/2) t(x)
• The usual notation to compute symmetrically
  around the column at hand
• Values required that are outside the signal

14
Boundary Conditions
Values required that are outside the image
Constant 0 0 0 0 0 0 0 0 0 0 0 0 1 2
3 0 0 2 3 4 0 0 3 4 5
Extend 1 1 1 2 3 1 1 1 2 3 1 1 1 2
3 2 2 2 3 4 3 3 3 4 5

• CC(c) N-1?x0 f(c x - N/2) t(x)

Periodic 3 4 2 3 4 4 5 3 4 5 2 3 1 2
3 3 4 2 3 4 4 5 3 4 5
Reflect 5 4 3 4 5 4 3 2 3 4 3 2 1 2
3 4 3 2 3 4 5 4 3 4 5
15
NCC Application 1 determine shift vector
16
NCC Application 2 compare images
17
NCC Application 3 locate feature
image

• Qualitative locate a at global peak
• Quantitative a-s can be found at 1
• Quantitative o-s can be found at 0.7

18
Several Ways to Normalise Cross-correlation
energy
3 J. Martin and J.L. Crowley. Experimental
comparison of correlation techniques. In Proc.
International Conf. on Intelligent Autonomous
Systems, 1995.
19
Fast NCC Implementation 4

• Numerator computed via FFT as a convolution with
  the template reversed

• FFT requires size 2N, pad with zeros
• FFT is periodic, prevent errors by padding larger
  area 5

4 J.P. Lewis. Fast normalized
cross-correlation. In Vision Interface, pages
120123, 1995. 5 H.Huang, D.Dabiri and
M.Gharib. On errors of digital particle image
velocimetry.Meas. Sci. Technol. 8 (1997)
1427-1440.
20
Fast NCC Implementation
image energy under template

• Denominator computed from table containing the
  integral(running sum) of the image square over
  the search area.

21
Fast NCC Implementation Integral Image
Def The integral image at location (x,y), is the
sum of the pixel values above and to the left of
(x,y), inclusive.
Using the integral image representation one can
compute the value of any rectangular sum in
constant time. For example the integral sum
inside rectangle D we can compute as ii(4)
ii(1) ii(2) ii(3)
6 P. Viola and M. Jones. Robust real-time
object detection.Second International Workshop
on Statistical and Computational Theories of
Vision, 2001.
22
Results Timing )

• While Analysing, Registrate and Correlate
• Spatial Domain NCC 40 minutes
• Fast NCC 300 ms

• While Measuring, Registrate (Preliminary)
• Decimate image to 64 x 64 pixels2
• Apply Gaussian sub-pixel interpolation 7
• Background subtraction plus fast NCC 14 ms

) timing for images of 512 x 512 pixels2 on a PC
with an AMD Athlon at 2.8 GHz
7 J. Bolinder. On the accuracy of a digital
particle image velocimetry system. 1999.
23
Results Stabilisation
Au(110) 300 K 39 x 38 nm 26 sec/frame
Au(110) 300 K 52 x 55 nm 3.8 sec/frame
24
Summary

• NCC enables finding features
• NCC enables quantitatively comparing features
  images
• NCC enables tracking to compensate for drift,
  there is room for improvement

Future improvement Lucas-Kanade 8

• Spatial intensity gradient
• Taylor series expansion, iteration
• Gaussian Filter (? ? resolution)
• Pyramid of images at different resolution

8 B. Lucas and T. Kanade, An iterative image
registration technique with an application to
stereo vision, in Proc. Imaging Understanding
Workshop, 1981, pp. 121130.
25
Recognizing FeaturesCoalescence of Vacancy
Islands on Cu(100)
Paul Ruijgrok
200 x 200 nm 300 K Cu(100)
26
Finding the Vacancy Islands
Paul Ruijgrok
27
Leveling the Image
Paul Ruijgrok

• Accuracy
• Data based number of bins
• Fit (part of) Gaussian curve

28
Finding the Vacancy Islands threshold
Paul Ruijgrok
hthreshold h0 sa0 , s 0.10.9
29
Detecting the Island Edges
Paul Ruijgrok
Island A Erosion E(A,N4) ?A AE(A,N4)

• erosion

30
Finding the Vacancy Lines
a 1, b 1 ? y x 1
Paul Ruijgrok
Hough Transform
yi axi b or b -xia yi Transform
points to curves in parameter space
9 Duda, R. O. and P. E. Hart, "Use of the Hough
Transformation to Detect Lines and Curves in
Pictures," Comm. ACM, Vol. 15, pp. 1115
(January, 1972).
31
Finding the Vacancy Lines
Paul Ruijgrok

• Hough Transform
• Slope-intercept representationunbounded
  parameters
• Want grid of limited size
• ? x cos(?) y sin(?) , or
• ? C cos(? d)

32
Summary
Paul Ruijgrok
Thanks to DIPimage team, Delft University of
Technology. DIPimage a scientific image
processing toolbox for MATLAB.
33
Thanks To
Staffprof.dr. J.W.M. Frenken (Joost)dr.ir. T.H.
Oosterkamp (Tjerk)dr. M.J. Rost (Marcel)
Ph.D. Studentsdrs. K. Schoots (Koen) Undergradua
te Students P.V. Ruijgrok (Paul)
Technicians L. Crama (Bert)E. van Tol-Homan
(Els) R. Koehler  (Raymond)P. Schakel (Peter)
www.LeidenProbeMicroscopy.com
34
Summary
Hough Transform
35
The Future Superresolution ?
Hough Transform
36
References

• 1 G. Binnig, H. Rohrer, C. Gerber, and E.
  Weibel, Phys. Rev. Lett. 49, 57 (1982).
• 2 M.J. Rost, L. Crama, P. Schakel, E. van Tol
  et al. Rev. Sci. Instrum. 76 (2005) 053710
• 3 J. Martin and J.L. Crowley. Experimental
  comparison of correlation techniques. In Proc.
  International Conf. on Intelligent Autonomous
  Systems, 1995.
• 4 J.P. Lewis. Fast normalized
  cross-correlation. In Vision Interface, pages
  120123, 1995.
• 5 H.Huang, D.Dabiri and M.Gharib. On errors of
  digital particle image velocimetry.Meas. Sci.
  Technol. 8 (1997) 1427-1440.
• 6 P. Viola and M. Jones. Robust real-time
  object detection. Second International Workshop
  on Statistical and Computational Theories of
  Vision, 2001.
• 7 J. Bolinder. On the accuracy of a digital
  particle image velocimetry system. 1999.
• 8 B. Lucas and T. Kanade, An iterative image
  registration technique with an application to
  stereo vision, in Proc. Imaging Understanding
  Workshop, 1981, pp. 121--130.
• 9 R. Duda and P. Hart. Use of the Hough
  transformation to detect lines and curves in
  pictures. Comm. ACM, Vol. 15, pp. 1115
  (January, 1972).

37
Other Information

• Du-Ming Tsai , Chien-Ta Lin, Fast normalized
  cross correlation for defect detection, Pattern
  Recognition Letters, v.24 n.15, p.2625-2631,
  November 2003
• Ian T. Young, Jan. J. Gerbrands and Lucas J. van
  Vliet. Fundamentals of Image Processing. 1998.
• W.H. Press, S.A. Teukolsky, W.T. Vetterling,
  B.P. Flannery. Numerical Recipes in C The Art
  of Scientific Computing, 2nd edition. Cambridge
  University Press. New York, NY, USA.
• Ullrich Köthe. STL-Style Generic Programming
  with Images. C Report Magazine 12(1), pp.
  24-30, January 2000.
• Leiden Probe Microscopy
• Interface Physics at Leiden University
• This presentation from authors web-site

38
Software

• Stan Birchfield. Dept. of Electrical and
  Computer Engineering. Clemson University.
• KLT An Implementation of the Kanade-Lucas-Tomasi
  Feature Tracker.
• Quantitative Imaging Group at the Faculty of
  Applied Sciences, Delft University of Technology.
  The Delft Image Processing library. 1999-2004.
• Quantitative Imaging Group at the Faculty of
  Applied Sciences, Delft University of Technology.
  DIPimage, A Scientific Image Processing Toolbox
  for MATLAB. 1999-2004.
• Insight Software Consortium. National Library of
  Medicine Insight Segmentation and Registration
  Toolkit (ITK). 1999-2003.
• Cognitive Systems Group, University of Hamburg,
  Germany. The VIGRA Computer Vision Library.
  1999-2005.
• Chair of Technical Computer Science, RWTH Aachen
  University. LTI-Lib library for image processing
  and computer vision. 1999-2003.

39
Testbeeld