Machine Vision Applications

1
Machine Vision Applications

• Handling Semi-flexible Objects Scissors

2
Physical Layout

• Overhead camera
• Scissors lie on a flat-topped table
• Scissors can be at any position and orientation
  but must lie entirely within the camera's field
  of view
• Scissors can be fully closed, partially open, or
  fully open

3
Sample Images(N.B. Euler number changes open,
-1 closed, -2)
4
Guiding a Robot Using Vision

• Start with a binary image of a pair of scissors,
  which may opened by an arbitrary amount (fully
  closed to fully open).
• The vision system locates and determines the
  orientation of the larger of the two finger hole.
  
• The robot places its gripper in the larger finger
  hole picks up the scissors.
• The robot closes the scissors by pressing one of
  the 2 "free" ends against an upright pillar.
• The robot packs the scissors in an appropriately
  shaped slot in a tray of tools.
• Using another camera, the vision system checks
  the final placement of the scissors.
• N.B. The vision system and the robot act
  together.
• This procedure works whatever the opening
  angle.

5
Larger Finger Hole
6
Determining whether the scissors are fully closed
Use Euler number (eul in QT) Equal to -1 if
scissors are open Equal to -2 if they fully
closed. N.B. The largest lake is always the
larger finger hole, whatever the opening angle.
7
Obtaining a Secure Grasp

• Locate the finger holes as these are invariate.
• Since we eventually need the scissors closed, we
  can place a gripper in the larger finger hole.
• Then, lift the scissors and press them against a
  bar, or post, to push them closed.
• The robot can "normalise" the scissors (i.e.
  close them). This is easier than relying on the
  vision system to compensate for the variable
  shape.

8
Locating the Larger Finger Hole (i.e. largest
lake)

• QT command sequence
• blf Fill lakes
• exr Exclusive OR
• big Retain only the biggest blob
• x,y cgr Position of blob centroid
• z lmi Orientation

9
Chirality

• Normalise the scissors (i.e close them)
• Draw a straight line from the centroid of larger
  finger holes to the centroid of the smaller
  finger hole (Use big(2) to isolate this.)
• Find whether the centroid of the scissors is to
  the left or right of this line.
• This defines the chirality arbitrarily but
  consistently.

10
Estimating the degree of opening

• Measure the distance between the centroids of
  the finger holes (lakes). This increases
  monotonically as the scissors are opened wider.
• When the blades are nearly closed, locating the
  tips is difficult when they are fully closed, it
  is impossible.
• Locating the centroids of the finger holes is
  easier and more reliable than locating the tips
  of the blades.

11
Chirality and Opening Angle
12
QT Command Sequence

• x0,y0 cgr Centroid of the scissors
• blf Fill lakes
• exr Exclusive OR
• big(1) Retain only the biggest blob
• x1,y1 cgr Position of centroid of largest
  blob
• swi Interchange images
• big(2) Retain only the second biggest blob
• x2,y2 cgr Centroid of 2nd largest blob
• s (x0-x1)(y2-y1) -(y0-y1)(x2-x1)
• s 0 corresponds to straight line
• from x1,y1 to x2,y2
• chirality sign(s) Is s gt 0?