33rd Annual Simulation symposium ANSS2000, April 16-20, 2000. Washington, D.C Samir Otmane E-mail : [e-mail] CEMIF, Laboratoire Syst

1
33rd Annual
Simulation symposium ANSS2000,
April 16-20, 2000. Washington, D.C
Samir Otmane
E-mail otmane_at_cemif.univ-evry.
fr CEMIF, Laboratoire Systèmes Complexes
Http http//lsc.cemif.univ-evry.fr8
080/otmane 40 Rue du Pelvoux 91020 Evry,
France Tél [phone]
Fax [phone]

• Active Virtual Guides as an Apparatus for
  Augmented Reality Based Telemanipulation System
  on the Internet

2
Contents

• Introduction
• ARITI System
• Virtual guides/fixtures
• Unified Formalism
• Simple and Complex Virtual fixtures
• Virtual guides representation
• Manipulation on the screen
• Deforming guides to generate an appropriate
  guides
• Experiments and Results
• Conclusion and perspectives

3
Introduction- Tele-work -

• Master site
  Communication support Slave
  site
• The slave site is distant from the master
  site.
• Information feedback is corrupted by a
  bandwidth limitation of communication support
  .
• Time delay is not constant when using any
  communication network.
• No portable and user-friendly Tele-work systems.
• Human performances are decreased during direct
  control of remote Tele-manipulation task.

Information feedback
Sending orders
4
ARITI system

• Virtual reality and Augmented Reality
  technologies are used to
• Overcome the instability of time delay,
• Complete or compensate the information feedback
  (video feedback for instance)
• JAVA programming Language is used to implement
  the Man Machine Interface of ARITI system to
• Give a portable system and
• User-friendly Tele-work system

5
Interaction between Human
and remote Task Environment

• During interaction control of a remote robotic
  terminal tool, the user must
• Perform a physical action to initiate motion from
  the robot,
• Wait for the system to respond,
• Perceive the physical effect onto the robot and
  task environment,
• Decide what to do next,
• Repeat the cycle until the task is completed.
• Motor activity is initiated through interaction
  with a software interface via Keyboard and mouse,
  joystick, master arm, etc...

N E T W
HARD
SOFT
Human Operator
Robotic Interface
Remote Robotic Environment
6
Interactions withA R I T I

• Three kinds of visual assistance are given to
  human operator for friendly human computer
  interaction using the ARITI interface. These
  visual helps are devoted to
• Environment perception
• Robot control
• Robot supervision

Perception
N E T W
HARD
Remote Environment
Human Operator
Control
Supervision
Robotic Interface
7
Assistance for Environment Perception
Several Virtual view points

Human Operator
Video image feedback
Perception module
In Control module
8
Assistance for Robot Supervision
Textual information of the current task

Human Operator
Overlaid Model / Image
Supervision module
9
Assistance for Robot Control
Virtual robot
Human Operator (HO)
Control module
In Supervision module
10
System description Hardware

• ARITI system is implemented on a PC Pentium 233
  Mhz with a 128 Mo RAM.
• The PC is equipped with a Matrox Meteor video
  acquisition card connected to a black and white
  camera.
• The orders are sent via the RS232 serial link.

Video acquisition
Orders
RS232 serial link
11
System description Software

• ARITI system is implemented under LINUX operating
  system.
• ARITI interface is written based on JAVA object
  programming language
• Video server is written using the C standard
  language.
• Control server is written using the C and ASM
  (Microprocessor Assembly Language )

S O K E T
Video Client
Video server
-ARITI- INTERFACE
Control Client
Control server
Applet JAVA
C and ASM
L I N U X - O S -
12
To use the ARITI system
WWW CLIENTS Internet Browser
A R I T I System
Camera
Robot
13
The ARITI Display
http//lsc.cemif.univ-evry.fr8080/Projets/ARITI
14
Question !!
How to increase Human Operator performances to
do Telemanipulation task very easier ?
15
Increase Assistance for Robot Control
Virtual robot

Human Operator
Virtual Fixtures
Control module
In Supervision module
16
Virtual Fixtures Structure
17
Simple Virtual Fixtures

• Human operator can create and use virtual
  fixtures to control the robot very easier.
• Some examples of simple Virtual Fixtures (VF)

Disc
Sphere
Plan
Super-ellipsoid
Cone
Cube or Square
Cylinder
Pipe
18
Complex Virtual Fixtures
Delimiting the workspace between two robots in
cooperation
Following an arbitrary trajectory by the robot
Reaching a dangerous target with the robot end
tool
19
Method of construction

• taking some significant points on the surface of
  the guide
• joining these points in order to get a wire frame
  representation
• Example
• If a parametric equation of the guide is
• with
• Then the vertexs are
• And the segments are

20
Manipulation onthe screen

• Use of the graphic camera model
• Is the matrix M which transform point
  coordinates (Xo, Yo, Zo ) In the referential Ro,
  onto the screen point coordinates (U, V) .

21
Selection of the 3D
Fixtures on the screen

• The designation on the screen 2D point (U, V)
• Determining what object 3D wanted to designate
• The designated point belongs to a D segment,
  witch equation is

22
Selection of the 3D Fixtures on
the screen

• Determining what vertex on the 3D objects having
  the smallest distance from the line segment D.

Selected fixture
Screen
Selected point
23
DeformingVirtual Fixtures

• Each virtual fixture is associated to a graph
• where X is a set of vertexs,
  and U a set of lines .
• We define an application V which associates any
  vertex x in X a set of his neighbors
• We call a distance between the vertex x and
  y.

24
DeformingVirtual Fixtures

• If x0 is a start point of deformation and
• ?0 the value of this deformation,
• Then the value of deformation of the fixture is
  given by
• Where p is called initial propagation factor
• and f is called the dissipation of propagation
  factor

25
DeformingVirtual Fixtures
P 0.99, f 0.99
P 0.99, f 0.9
P 0.99, f 0.9
26
Experiments

• Pick and place task
• Tele-operation mode
• Control the real robot via the virtual robot
• 10 human operators (HO)
• 3 kinds of test
• Without Virtual Fixtures
• With passive Virtual Fixtures
• With active (attractive) Virtual Fixtures
• Each HO makes 10 tests for each kind

27
Experiments Task Board

• The robot is assumed to assemble (place) and
  disassemble (pick) objects hanging on a metal
  stand

Target
Objects
Head of the robot peg
Metal stand
28
Fixture to reacha target

• A simple geometric primitive
• Cone .

29
Fixture to pick theobject and unhook it

• A complex Virtual guide
• Combining a 3 simple guides (cylinders)

30
Fixture to place the object onto the stand

• A complex Virtual guide
• Combining 4 simple guides ( 1 cone 3 cylinders)

31
ResultsReach a cylinder N 1

• Reach a 3D target point on the peripheral of the
  cylinder N1
• Without virtual fixtures there is 1.49 collision
  for each test

32
ResultsReach a cylinder N 1
Average time 7,7 sec with active VFs
Imprecision errors on X, Y, Z axis Err lt 0,25
mm with active VFs
33
Results Pick and place a cylinder N 1

• - Blue - with passive VFs.
• - Red - with active (attractive potential
  fields) VFs.

Unhook a cylinder N 1 Passive VFs -
12,78 sec Active VFs - 9,5 sec

Place a cylinder into the stand, Passive VFs -
37,96 sec Active VFs - 7,86 sec
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Conclusion

• Thanks to Virtual Fixtures the human operator
  performances are increased
• best accuracy lt 0,25 mm
• best completion time
• best safety
• JAVA programming Language is used to implement
  these Virtual Fixtures into the ARITI system to
• Give a portable and interactive fixtures.

38
Perspectives

• Use the Virtual Fixtures for mobile robot
  application, such as
• Navigation, obstacles avoidance, to assist
  disable person.

39
Perspectives

• Add Tactile functions to activate fixtures
• Extend the use of virtual fixtures to do a
  cooperative Tele-Work

User 1
Robot 1
User 2
N E T
. . .
Robot 2
Extended A R I T I System
. . .
User n
Robot m