Vision Guided Robotics - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Vision Guided Robotics

Description:

... 3-D Positioning of Surgical Instruments in Robotized Laparoscopic Surgery Using Visual Servoing ... make more intelligent 'surgical assistants' possible. ... – PowerPoint PPT presentation

Number of Views:325
Avg rating:3.0/5.0
Slides: 33
Provided by: unkn744
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Vision Guided Robotics


1
Vision Guided Robotics
  • and Applications in Industry and Medicine
  • Matthias Rüther

2
Contents
  • Robotics in General
  • Industrial Robotics
  • Medical Robotics
  • What can Computer Vision do for Robotics?
  • Vision Sensors
  • Issues / Problems
  • Visual Servoing
  • Application Examples
  • Summary

3
Robotics
  • What is a robot?
  • "A reprogrammable, multifunctional manipulator
    designed to move material, parts, tools, or
    specialized devices through various programmed
    motions for the performance of a variety of
    tasks"
  • Robot Institute of America, 1979
  • Industrial
  • Mostly automatic manipulation of rigid parts with
    well-known shape in a specially prepared
    environment.
  • Medical
  • Mostly semi-automatic manipulation of deformable
    objects in a naturally created, space limited
    environment.
  • Field Robotics
  • Autonomous control and navigation of a mobile
    vehicle in an arbitrary environment.

4
Robot vs Human
  • Robot Advantages
  • Strength
  • Accuracy
  • Speed
  • Does not tire
  • Does repetitive tasks
  • Can Measure
  • Human advantages
  • Intelligence
  • Flexibility
  • Adaptability
  • Skill
  • Can Learn
  • Can Estimate

5
Industrial Robot
  • Requirements
  • Accuracy
  • Tool Quality
  • Robustness
  • Strength
  • Speed
  • Price Production Cost
  • Maintenance

Production Quality
6
Medical (Surgical) Robot
  • Requirements
  • Safety
  • Accuracy
  • Reliability
  • Tool Quality
  • Price
  • Maintenance
  • Man-Machine Interface

7
What can Computer Vision do for Robotics?
  • Accurate Robot-Object Positioning
  • Keeping Relative Position under Movement
  • Visualization / Teaching / Telerobotics
  • Performing measurements
  • Object Recognition
  • Registration

Visual Servoing
8
Vision Sensors
  • Single Perspective Camera
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)
  • Laser Scanner
  • Omnidirectional Camera
  • Structured Light Sensor

9
Vision Sensors
  • Single Perspective Camera

10
Vision Sensors
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)

11
Vision Sensors
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)

12
Vision Sensors
  • Laser Scanner

13
Vision Sensors
  • Laser Scanner

14
Vision Sensors
  • Omnidirectional Camera

15
Vision Sensors
  • Omnidirectional Camera

16
Vision Sensors
  • Structured Light Sensor

Figures from PRIP, TU Vienna
17
Issues/Problems of Vision Guided Robotics
  • Measurement Frequency
  • Measurement Uncertainty
  • Occlusion, Camera Positioning
  • Sensor dimensions

18
Visual Servoing
  • Vision System operates in a closed control loop.
  • Better Accuracy than Look and Move systems

Figures from S.Hutchinson A Tutorial on Visual
Servo Control
19
Visual Servoing
  • Example Maintaining relative Object Position

Figures from P. Wunsch and G. Hirzinger.
Real-Time Visual Tracking of 3-D Objects with
Dynamic Handling of Occlusion
20
Visual Servoing
  • Camera Configurations

End-Effector Mounted
Fixed
Figures from S.Hutchinson A Tutorial on Visual
Servo Control
21
Visual Servoing
  • Servoing Architectures

Figures from S.Hutchinson A Tutorial on Visual
Servo Control
22
Visual Servoing
  • Position-based and Image Based control
  • Position based
  • Alignment in target coordinate system
  • The 3D structure of the target is rconstructed
  • The end-effector is tracked
  • Sensitive to calibration errors
  • Sensitive to reconstruction errors
  • Image based
  • Alignment in image coordinates
  • No explicit reconstruction necessary
  • Insensitive to calibration errors
  • Only special problems solvable
  • Depends on initial pose
  • Depends on selected features

End-effector
target
Image of end effector
Image of target
23
Visual Servoing
  • EOL and ECL control
  • EOL endpoint open-loop only the target is
    observed by the camera
  • ECL endpoint closed-loop target as well as
    end-effector are observed by the camera

EOL
ECL
24
Visual Servoing
  • Position Based Algorithm
  • Estimation of relative pose
  • Computation of error between current pose and
    target pose
  • Movement of robot
  • Example point alignment

p1
p2
25
Visual Servoing
  • Position based point alignment
  • Goal bring e to 0 by moving p1
  • e p2m p1m
  • u k(p2m p1m)
  • pxm is subject to the following measurement
    errors sensor position, sensor calibration,
    sensor measurement error
  • pxm is independent of the following errors end
    effector position, target position

26
Visual Servoing
  • Image based point alignment
  • Goal bring e to 0 by moving p1
  • e u1m v1m u2m v2m
  • uxm, vxm is subject only to sensor measurement
    error
  • uxm, vxm is independent of the following
    measurement errors sensor position, end effector
    position, sensor calibration, target position

p1
p2
u1
v1
v2
u2
d1
d2
c1
c2
27
Visual Servoing
  • Example Laparoscopy

Figures from A.Krupa Autonomous 3-D Positioning
of Surgical Instruments in Robotized Laparoscopic
Surgery Using Visual Servoing
28
Visual Servoing
  • Example Laparoscopy

Figures from A.Krupa Autonomous 3-D Positioning
of Surgical Instruments in Robotized Laparoscopic
Surgery Using Visual Servoing
29
Registration
  • Registration of CAD models to scene features

Figures from P.Wunsch Registration of CAD-Models
to Images by Iterative Inverse Perspective
Matching
30
Registration
  • Registration of CAD models to scene features

Figures from P.Wunsch Registration of CAD-Models
to Images by Iterative Inverse Perspective
Matching
31
Tracking
  • Instrument tracking in laparoscopy

Figures from Wei A Real-time Visual Servoing
System for Laparoscopic Surgery
32
Summary
  • Computer Vision provides accurate and versatile
    measurements for robotic manipulators
  • With current general purpose hardware, depth and
    pose measurements can be performed in real time
  • In industrial robotics, vision systems are
    deployed in a fully automated way.
  • In medicine, computer vision can make more
    intelligent surgical assistants possible.
About PowerShow.com