AM23

1

• AM23
• Lectures 12 Inspection, scanning, pose
  estimation and classification of 3D objects
• Contents
• Measurement, inspection, pose estimation and
• classification
• Achieving 3D information, why and how?
• Stereo vision
• Assistance of structured light
• Model based recognition
• Image based recognition

2

• Measurement and inspection
• Commercially available system for measurement and
  inspection
• checking electronic boards
• checking the content of multicomponent packs
• detecting impurities in medical products
• verifying labels on drug cans and bottles
• counting products with bar codes on conveyer
• belts
• assessing volumes (food industry)
• assessing textures and colours (cloth and fur)
• etc., etc.

3

• Robot grasping or surface treatment of industrial
  products
• Automation of production of small series of
• products requires flexible work cells.
• The feeding of products and components to a
• flexible work cell is a fundamental problem.
• The reason is the high price and low flexibility
  
• of fixtures and vibrating bowls necessary for
  blind robots and other automatic equipment.
• There is a strong need for vision systems
  capable of
• guiding robots to grasp components lying or
  hanging
• disordered

4

• Agriculture, green house production, food
  industry
• Areas of potential vision applications in
  producing food and flowers
• Meet processing
• Weed removal on field without using pesticides
• Fruit picking
• Handling of cuttings in flower industry
• etc.

5

• Achieving 3D information
• Structured light
• A beam of laser light (a single spot)
• Oscillatory scanning of a laser beam
• A sheet of laser light (line focused laser)
• A grid of laser spots or lines
• Projection of other patterns
• Stereo vision
• A priori knowledge on object size
• Model based recognition
• Image based recognition
• Range scanner (time-of-flight)

6

• Line focussed lasers
• Examples
• Volume measurements of fish fillets
• Hanging objects
• Angle measurements in machine
• bending

7
Volume measurements of fish fillets
8
Hanging radiator on its way to a powder paint
process
9
Angle measurements in bending machines
10
Stereo vision

P is the object point C and C are pinholes of
the two cameras u and u are images of P
The line through u and e is the epipolar line in
the left camera associated with the image point
u of the right camera. This 2D line is the image
in the left camera of the 3D line X
11
Stereo vision

• Three steps in stereo vision
• Rectification using a homographic)
  transformation of the images
• Finding correspondences along epipolar lines
  thereby finding the disparities D
• 3D reconstruction using Z 2hf/D
• ) This homographic transformation corresponds to
  a virtual rotation of the cameras about their
  pinhole. If the focal lengths are unequal, also a
  scaling is involved. The new image can be
  generated by a pixel coordinate transformation.

12
Stereo vision

Image rectification Image transformation so that
1) Focal lengths are equal 2) Optical axes are
parallel 3) Epipolar line is a pixel row P is a
3D point Pl and Pr are images of P 2h is the
baseline f is the focal length
The disparity D is defined as PlOl PrOr From
2D geometry Z 2hf/D
13
Geometry of rectification of stereo images

Pha and Phb are camera pinholes. Planes P a and P
b are planes perpendicular to the baseline. P is
the pseudointersection of the optical axes. Qa
and Qb are the projections of P on the planes P a
and P b. PhaQa and PhbQb are new optical axes.
14
Model based pose estimation

15
Model based pose estimation

The 2D points corresponding to some specific 3D
points are found in the image How to estimate
the 3D pose of the object? How many
corresponding points are necessary?
16
Try three points The three points are lying on
known rays through the pin hole. Unknowns Three
distances from the pinhole to the 3D
points Equations The lengths of three sides in
the triangle spanned by the three 3D points
can be expressed in terms of the above
distances. The lengths are also known from
the physical object. 3 equations and 3
unknowns The equations are quadratic, so the
equations may have 6 mathematical solutions.
Usually, only one or two are real and physically
relevant. Iterative numerical methods must be
employed in solving the 3 equations.
17
View-based recognition During training Recording
off-line of many training images and derivation
of features During recognition Comparison between
features of a recognition image with those of
training images Which features? Choice between
or combination of 1) template match (edges or
regions) and 2) comparison between numerical
features With numerical features Choice between
partial features and features requiring complete
segmentation
18
View-based recognition What are partial
features of contours?
19

• View-based recognition
• Problem
• Position and orientation (pose) of the camera
  relative to the object depend on 6 degrees of
  freedom
• A choice of 6 variables
• Pinhole position (r sin j cos q, r sin j sin q, r
  cos j) in the frame of the object
• Camera rotation about pinhole (Tilt, Pan, Roll)
• A. Weak perspective
• Changes of Tilt and Pan corresponds to 2D
  translation,
• Roll corresponds to 2D rotation
• Change of r corresponds to 2D scaling
• Training by variation of (q , j ) only

20
View-based recognition
B. Strong perspective

• Changes of Tilt and Pan give complicated, but
  predictable image changes)
• Roll corresponds to 2D rotation
• Change of r gives changes which are not a simple
  2D scaling)

Training with variation of (r, q, j) ) See next
slide ) See slide after next
21
View-based recognition
Concerning complicated, but predictable changes
22
View-based recognition Changes of the variable r
does not correspond to 2D scaling in case of
strong perspective
In case of an object far away, parallel 3D lines
on the object give parallel image
lines. Close-up images lead to non-parallel
image lines of parallel 3D lines