Robotics, Intelligent Sensing and Control Lab RISC

1
Robotics, Intelligent Sensing and Control Lab
(RISC)
University of Bridgeport School of Engineering
Tarek SobhVice president for graduate studies
and research Dean, School of Engineering
2
Outline of Outgoing Project

• Online Automation and Control An Experiment in
  Distance Engineering Education
• E-Learning Case Studies in Web-Controlled
  Devices and Remote Manipulation
• Prototyping Environment for Robot Manipulators
• Manipulator Workspace Generation and
  Visualization in the Presence of Obstacles
• Kinematic Synthesis of Robotic Manipulators from
  Task Descriptions
• New concept in optimizing the manipulability
  index of serial Manipulators using SVD method

3
Outline of Outgoing Project

• Industrial Inspection and Reverse Engineering
• Recovering 3-D Uncertainties from Sensory
  Measurements for Robotics Applications
• Sensing Under Uncertainty for Mobile Robots
• Service RobotsA Tire Changing Manipulator
• Robot Design and Dynamic Control Simulation
  Software Solutions From Task Points Description
• Experimental Robot Musicians
• Design and Implementation of a Multi-sensor
  Mobile Platform

4
Online Automation and Control An Experiment in
Distance Engineering Education
5
Introduction

• Online Distance Education is a major part of the
  current education system
• Started as an internal exercise to share and
  discuss ideas
• Ever growing need for part-time education
• 213 Universities offering online courses at
  various levels and disciplines in the US
• Majority of the online courses are non-technical
• Lacking laboratory based courses

6
Need for Online Education

• Part time course work
• Working class willing to pursue higher education
• Social responsibilities
• Current socio-political situation
• National and International demand

7
Distance Engineering Education

• Accredited engineering degrees
• Under-graduate and Graduate level
• Computer Engg, Electrical Engg, Mechanical Engg
• Comprehensive laboratory based courses.

8
Partnerships

• Great value of American engg. degrees overseas
• Partnership with foreign University/Institution
  providing
• Infrastructure
• Teaching support
• Examination facilities
• Closer to the student concentration
• Helps in better delivery of courses

9
Projects Implemented Towards DL Education

• Mobile Robot Controlled by a Phone
• Internet Based Software Library for the SIR-1
  Serial Port Controlled Robot
• Internet Based Computer Vision Framework For
  Security, Surveillance And Tracking Applications

10
Online Distance Laboratories

• Using Automation and Telerobotic (controlling
  devices from a distance) systems
• Real-time laboratory experience via the internet
• Tele-operation of Mitsubishi Movemaster
• RISCBOT A Web Enabled Autonomous Navigational
  Robot
• Tele-operation of the FESTO Process Controller

11
1. Tele-operation of Movemaster

• Can be used in 3 modes
• Evaluation mode
• Teacher mode
• Student mode

12
RISCBOT

• Waits for command from the server.
• Wall clinging robot.
• Image processing program checks for doors.
• Uses Ultrasonic sensors for obstacle avoidance.
• PC acts as central decision maker.

13
RISCBOT IN ACTION
14
RISCBOT CONTROL WEBSITE
15
FESTO Process Controller

• Providing telerobotic operability of the FESTO
  process control machine by interfacing it with
  the Mitsubishi Movemaster robot.

16
Conclusion

• Virtual online collaboration
• Lab-based distance education
• Accredited Engineering/Technical lab-based
  experience, degrees training via distance
  learning

17
E-Learning Case Studies in Web-Controlled
Devices and Remote Manipulation
18
Mobile Robot Controlled by a Phone
An application of a Robot with a phonechip
19
Mobile Robot Controlled by a Phone
PHONEBOT Basic Block Diagram
20
Internet Based Software Library for the SIR-1
Serial Port Controlled Robot

• Web Based Control / Remote Automation
• API functions for SIR-1 Remote Manipulation
  direct / inverse kinematics, multiple
  simultaneous serial-port-communication
  interfacing, link speed control

21
Internet Based Software Library for the SIR-1
Serial Port Controlled Robot


Internet
22
Internet Based Computer Vision Framework For
Security, Surveillance And Tracking Applications

• Vision Framework for Real-Time Tasks with
  Off-The-Shelf Hardware
• Early processing (Gaussian Filters, Histogram
  Normalization, Color Filtering)
• Feature Extraction (Edge Detection, Line /
  Ellipse detection, Region Growing, Region
  Splitting, MinMax point extraction)
• Feature Matching

23
Internet Based Computer Vision Framework For
Security, Surveillance And Tracking Applications
acquisition
color filtering
conversion to monochrome
Gaussian blur
thresholding
MinMax feature extractor
heuristic feature detection
feature matcher
match result
24
Prototyping Environment for Robot Manipulators
25
Robot Prototyping Environment
26
Design Parameters Subsystem Notification
27
Database Design Considerations
28
To design a robot manipulator, the following
tasks are required

• Specify the tasks and the performance
  requirements.
• Determine the robot configuration and parameters.
• Select the necessary hardware components.
• Order the parts.
• Develop the required software systems
  (controller, simulator, etc...).
• Assemble and test.

29
Web Enabled Robot Design and Dynamic Control
Simulation Software Solutions From Task Points
Description
30
Research Summary

• A web-based solution for robot design and dynamic
  control simulation based on given task point
  descriptions
• The software combines and utilizes the
  computational power of both the Mathematica and
  Matlab packages

31
Research Summary (cont.)

• Given the location and velocity of each task
  point, our approach formulates the complete
  design of a 3 DOF robot model by computing its
  optimal dynamic parameters such as link length,
  mass and inertia
• Suggests the optimal control parameters (Kp, Kv)
  for the dynamic control simulation

Puma560 3 DOF robot
32
The Software Package

• Web Interface
• Kinematic Design Module
• Dynamic Design Module
• Dynamic Control Simulation Module

33
The Software Package (Cont.)
34
Web Interface

• JSP, Servlet, JLink and JMatservlet
• Central control module

35
Results User login Screen
sample run video
36
User specifies number of task points
37
User specifies the coordinates and velocities of
each task points with respect to a time scale
38
User specifies link radii for dynamic model
generation, and Kp, Kv initialization for dynamic
PD control simulation
39
DH table, Dynamic Parameter Matrix and optimal
Kp, Kv values for each link
40
A standard PPP model
41
Desired Trajectory for link 1, 2, 3
Desired Vs. obtained link displacement for link 1
42
Desired Vs. obtained link displacement for link 2
Desired Vs. obtained link displacement for link 3
43
Desired velocity trajectory for link 1, 2 and 3
Desired Vs. Obtained velocity for link 1
44
Desired Vs. Obtained velocity for link 2
Desired Vs. Obtained velocity for link 3
45
Ergonomic and Efficient Software
Alternatives for High Cost Manipulators - Direct,
Wireless and Networked Control Techniques
46
High Cost Manipulators

• deciding-on and purchasing the right
  manipulator(s) for a predetermined task (budget,
  purchasing time)
• educational institutions (diversity of software /
  hardware controlling techniques possibility of
  becoming victims of abusive usage)

47
Ergonomic and Efficient Software Alternative

• software simulation and control package
• standalone simulator
• networked simulator
• virtual manipulator
• remote automation / distance learning
• cell phone based control

48
The Manipulator Used in the Implementation

• Mitsubishi, RV-M1 (Movemaster EX)
• general purpose commercial arm
• 5 DOF

49
The Simulator Kinematics
IK/DK control, workspace-safe, real-time,
CAD/robot
50
The Simulator Trajectory Control
real-time trajectory modeling and testing,
CAD/robot
51
The Simulator Networking Model
client simulator
client/server simulator
client/server simulator
server/robot simulator

• direct serial link connectivity
• pipelined TCP/IP connectivity, allowing for
  effective distance learning methods and flexible
  remote automation and control

actual robot
52
The Simulator Networking Model Scenario
controlling the robot through 2 pipelined
simulators
53
The Simulator Cell Phone Server Mode
in cell phone server mode, the simulator allows
direct control over the manipulator or a pipeline
of simulators, through a web enabled cell phone
54
Manipulator Workspace Generation and
Visualization in the Presence of Obstacles
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
Kinematic Synthesis of Robotic Manipulators from
Task Descriptions
60
Envisioning Optimal Geometry
61
Objectives

• Parameters considered in this work
• Coordinates of the task-points
• Spatial constraints
• Restrictions (if any) on the types of joints
• Goals
• Simplified interface
• Performance
• Modular architecture to enable additional
  optimization modules (for velocity, obstacles,
  etc.)

62
Manipulability Measure
wvdet(JJT)

• For performance purposes the manipulability
  measure was the one originally proposed by Tsuneo
  Yoshikawa
• Singular configurations are avoided by maximizing
  the determinant of the Jacobian matrix

63
Optimization Measure
64
Screenshots
65
Sample I Trajectory
66
Sample I Manipulability Ellipsoids
67
(No Transcript)
68
(No Transcript)
69
Sample II Manipulability Ellipsoids
70
(No Transcript)
71
(No Transcript)
72
New concept in optimizing the manipulability
index of serial manipulators using the SVD method
73

• Studying the manipulability index for every point
  within the workspace of any serial manipulator is
  considered one of the important issues, required
  for designing trajectories or avoiding singular
  configurations.
• The manipulability measure is an indicator of how
  close the manipulator is to being in singular
  configurations .

74
Manipulability Bands of six degrees of freedom
manipulator
75
Manipulability Bands of Puma 560 in 2-D workspace
76
Manipulability Bands of Mitsubishi movemaster in
2-D workspace.
77
Industrial Inspection and Reverse Engineering
78
Why reverse engineering?

• Applications
• Legal technicalities.
• Unfriendly competition.
• Shapes designed off-line.
• Post-design changes.
• Pre-CAD designs.
• Lost or corrupted information.
• Isolated working environment.
• Medical.
• Interesting problem
• Findings useful.

79
Closed Loop Reverse Engineering
80
A Framework for Intelligent Inspection and
Reverse Engineering
81
Recovering 3-D Uncertainties from Sensory
Measurements for Robotics Applications
82
Propagation of Uncertainty
83
Flow Envelopes
84
Tolerancing and Other Projects
85
Problem
A unifying framework for tolerance
specification, synthesis, and analysis across the
domains of industrial inspection using sensed
data, CAD design, and manufacturing.
86
Solution
We guide our sensing strategies based on the
manufacturing process plans for the parts that
are to be inspected and define, compute and
analyze the tolerances of the parts based on the
uncertainty in the sensed data along the
different tool paths of the sensed part.
87
(No Transcript)
88
Sensing Under Uncertainty for Mobile Robots
89
Abstract Sensor ModelWe can view the sensory
system using three different levels of abstraction

• Dumb Sensor returns raw data without any
  interpretation.
• Intelligent Sensor interprets the raw data into
  an event.
• Controlling sensor can issue commands based on
  the received events.

90
Trajectory of the robot in a hallway environment
91
Trajectory of the robot in the lab environment
92
Potpourri of other RA Projects
93
Projects

• Modeling and recovering uncertainty in 3-D
  structure and motion
• Dynamics and kinematics generation and analysis
  for multi-DOF robots
• Active observation and control of a moving agent
  under uncertainty
• Automation for genetics application
• Manipulator workspace generation in the presence
  of obstacles
• Turbulent flow analysis using sensors within a
  DES framework

94
Service RobotsA Tire Changing Manipulator
95
Design and Construction

• A prototype of the racing car

96
Design and Construction

• The manipulator will be of the depicted form. The
  design was derived from inertial and dexterity
  calculations
• Three essential Components the sliding
  mechanism, the arm, and the end effector system.

97
Design and Construction

• All of the four arms should be suspended with the
  visualized sliding mechanism.

98
Complete Design Schema
customer
manipulator based manipulator manufacturing
web based interface
simulation based quality assurance and testing
data acquisition
prototyping environment
packaging and shipping
99
Experimental Robot Musicians
100
Introduction

• Robot musicians perform on real instruments
  through the usage of mechanical devices, such as
  servomotors and solenoids
• Research innovations linking music, robotics and
  computer science

101
MotivationMusic Expressiveness

• Offer the audience live-experience very similar
  to listening to a human musician.
• Real instrument performance, such as the physical
  vibration of a violin string, provides a much
  stronger case in music expressiveness, versus
  electronic music synthesizers.
• Mozart - eine kleine nacht musik whole ensemble

102
MotivationMusic Expressiveness (cont.)

• Bypass several technical difficulties that are
  typically encountered by human musicians
• More degrees of freedom in real-time performances
  and reach a higher level of performance
  difficulty, flexibility and quality.
• As an example, a violin is played by a robot
  musician with hands that have 12 fingers.

103
Robot Musicians ArchitectureRobot Musicians Band
Overview (cont.)

• Robot musicians, the P.A.M. band, invented by
  Prof. Kurt Coble.

The moth features violin solo, composed by Prof.
Kurt Coble, companied by percussion ensemble,
electric base and electric guitar
104
Robot Musicians ArchitectureRobot Musicians Band
Overview (cont.)
Austin plays a Percussion Ensemble
Dusty plays a red electric guitar
105
Robot Musicians ArchitectureRobot Musician
Architecture Overview

• A three-module architecture

106
Robot Musicians ArchitectureMotion Module (Cont.)
Servo attached to one bow of Jasche
Solenoid (with holding power of 1.5 pounds)
attached to Jasche
107
Servomotor In Action
108
Motion Module In Action

• Mozart - eine kleine nacht musik whole ensemble

109
Jasche In Action
Amazing grace traditional American folk song
110
Real Time PerformanceMechanical Issues
111
Results drum set in action
112
Robot Musicians ArchitectureMotion Module (Cont.)
A coffee containers plastic lid is connected
with a servo so it flutters against the body of a
drum when the servo receives control command from
the control module
Sample Motion Module Architecture drumstick
controlled by solenoid
113
Robot Musicians ArchitectureMotion Module (Cont.)
Sample Motion Module Architecture chimes wand
controlled by servo
114
Design and Implementation of a Multi-sensor
Mobile PlatformRISCBot II
115
Sensor Fusion

• Sensor fusion is a method for conveniently
  combining and integrating data derived from
  sensory information provided by various and
  disparate sensors, in order to obtain the best
  estimate for a dynamic systems states and
  produce a more reliable description of the
  environment than any sensor individually.

116
Sensor Fusion Categories

• Complementary sensors consist of sensors with
  different modalities, such as a combination of a
  laser sensor and a digital camera.
• Competing sensors are composed of sensors suit
  which have the same modality, such as two digital
  cameras which provide photographic images of the
  same building from two different viewpoints.

117
Data Acquisition

• we used a data acquisition module called Data
  Translation DT9814
• Advantages
• Low cost USB data acquisition module.
• 24 analog inputs, 2 analog outputs, and one
  32-bit counter timer .
• Analog signal range of /- 10V .
• Resolution of 12 bits for both the analog input
  and analog output subsystems, and input
  throughputs up to 50 kHz.

118
Sonar Sensor

• we used LV MaxSonar EZ0 ultrasonic sensors

119
MaxSonar -EZ0 Sensors

• They are low cost sonar ranger actually
  consisting of two parts
• An emitter, which produces a 42kHz sound wave
• A detector, which detects 42kHz sound waves and
  sends an electrical signal back to the
  microcontroller.
• Readings can occur up to every 50 ms, (20-Hz
  rate) and designed for indoor environments.
• Advantage
• They can detect obstacles with high confidence
  especially when the object is well defined

120
Infrared Proximity Sensor

• Infrared sensors operate by emitting an infrared
  light, and detecting any reflection off surfaces
  in front of the robot. If the reflected infrared
  is detected, it means that an object is detected.
• We have used an infrared proximity - Sharp
  GP20A21YK

121
Infrared Proximity Sensor
122
Jazzy 1122 Wheelchair
123
Jazzy 1122 Wheelchair
124
Navigation and Obstacle Avoidance

• The obstacle may be defined as any object that
  appears along the mobile robots.
• In the navigation problem, the requirement is to
  know the positions of the mobile robot and a map
  of the environment (or an estimated map).
• The ability of the robot to act based on its
  knowledge and sensor values so as to reach its
  goal positions as efficiently and as reliably as
  possible.

125
IMPLEMENTATION AND RESULTS
126
RISCbot II

• Reverse engineering process.
• Different types of sensors
• LV MaxSonar- EZ0 ultrasonic
• Sharp GP20A21YK infrared proximity sensors

127
RISCbot II
128
RISCbot II
129
Features

• RISCBot II is a high-payload platform with a
  payload up to 350 lbs.
• RISCBot II is a high-speed platform which moves
  up to 6 mph .
• RISCBot II powerful motors and two 14 pneumatic
  wheels on steel frame with suspension is designed
  for higher speeds with good response.

130
Features

• RISCBot II is equipped with Active-Trac
  Suspension (ATS).
• ATS makes the platform to traverse different
  types of terrain and obstacles while maintaining
  smooth operation .
• RISCBot II has two front Anti-Tip Wheels which
  work with the ATS to maneuver over obstacles .
• RISCBot II also has Rear Casters wheels to
  respond to the weight transfer and to pivot while
  driving over obstacles.

131
CT Post
132
CT Post
133
Research Collaborators

• Raul Mihali.
• Anatoli Sachenko.
• Sarosh Patel.
• Bei Wang.
• Puneet Batra.
• Amit Singh.
• Sudip Pathak.
• Tomas Vitulskis.
• Andrew Rosca.
• Ayssam El Kady.
• Eslam M. Gebriel.
• Mohammed Mohammed.

134

• Thank you