If%20you%20have%20any%20questions%20please%20feel%20free%20to%20interrupt%20me

1
If you have any questions please feel free to
interrupt me
2
The vision system for Marie Curie
3
Main Tasks in our system
Image recognition
Machine Learning
Control of robots behavior
Environment
4
Marie Curie will be communicating with
Schroedingers Cat robot
5
Interaction of Marie Curie and Cat

1. Marie Curie does not change its main
   coordinates, she can only move her hands, head
   and legs, but she remains attached to the desk.
2. Cat can move freely in the area of the stage.
3. Cat should not bump into Marie or furniture of
   the lab.
4. Marie should know where the cat is located and
   look to him
5. Cat should know where Marie is located and talk
   to her.

6
Ideal view of the ceiling camera
Black curtain
Cat
shelf
equipment
equipment
Marie Curie
equipment
7
There will be a Kinect camera looking from the
ceiling to the stage
8
The role of the ceiling camera

• The camera will be attached to the ceiling or
  will be in some position very high, as high as we
  can.
• We have done something similar but the robots
  were small.
• The camera should know x,y coordinates of every
  robot and its orientation (pose)
• Marie Curie does not change its main coordinates,
  so it is easy
• Cat is fast so we have to track the triplet
• (x, y, ?)

9
There is nothing like that in Disneyland

• All behaviors of robots in commercial theatres
  are strictly scripted, robots move on rails, they
  cannot make an error.
• In our case we have interaction, improvisation,
  and robots are subject to noisy behaviors.
• This task is more similar to robot soccer than to
  existing robot theatres in the world.

10
There will be another camera looking to faces of
the audience
We will call it the human-control camera or a
front camera
11
The role of the front camera

1. The camera is attached to the wall near the
   glass window of the theatre, looking towards
   humans located in the corridor.
2. This camera will look at the audience
3. There are several goals of having this camera
4. Recognizing (x, y, ?) of every person that looks
   at the performance (perhaps not more than 5).
5. Recognizing the emotion on the faces of these
   people.
6. Recognizing their gestures with hands and legs,
   full bodies and faces.
7. Use these data to control the behavior of the
   robots, songs selected, slides selected, lights
   and other effects.

12
There is nothing like that in Disneyland

• All behaviors of robots in Disneyland are
  strictly scripted.
• Rarely humans can change robots behaviors.

13
This is a new task for our team

• Marek Perkowski has never done anything like this
  before
• Perhaps nobody in the world has done something
  like this.
• This is good as we are doing something new.
• Hopefully we have done something similar and have
  a good experience from the past.
• We were doing ROBOT SOCCER.
• We will try now to use our past experience and
  theory for a new task.

14
Ideal view of the ceiling camera
Black curtain
Cat
shelf
equipment
equipment
Marie Curie
equipment
15
This is ideal, in reality the image will be much
distorted with noise and lightning and geometry
Y axis
Black curtain
?c
Yc
Cat
Xc
shelf
equipment
equipment
Marie Curie
equipment
X axis
16
The idea of Robot Soccer
17
3. Robot Soccer and Similar Tasks

• Robot Soccer Competition
• RoboCup
• FIRA
• Remote controlled systems
• Autonomous robots
• Clustering

18
3.1 Robot Soccer

• RoboCup is an international joint project to
  promote AI,
• robotics, and related fields.
• It is an attempt to foster AI and
• intelligent robotics research by providing a
  standard problem
• where a wide range of technologies can be
  integrated and
• examined.
• RoboCup chose to use the soccer game as a central
• topic of research, aiming at innovations to be
  applied for
• socially significant problems and industries.
• The ultimate goal
• of the RoboCup project is By 2050, develop a
  team of fully
• autonomous humanoid robots that can win against
  the human
• world champion team in soccer. RoboCup 1998

19
(No Transcript)
20
As you see, it is difficult to approximate every
robot with a rectangle. It will be even more
difficult in our case.
21
(No Transcript)
22
Overhead Vision

• Our goal is to start with Overhead Vision
  (Ceiling Camera) and check how it will work.
• We may move to more cameras if necessary.

23
Local Vision
24
Design Criteria for robot soccer

• Controller Hardware Enable on-board image
  processing
• Interface to digital camera
• Incorporate graphics LCD
• Incorporate user buttons
• Wireless communication between robots
• Sensors Allow variety of additional sensors
• Shaft encoders
• Infra-red distance measurement sensors
• Compass module
• Software Flexibility to accommodate for
  different robot equipment
• Operating system RoBIOS
• Hardware description table HDT

25
What is AI?

• Research in Artificial Intelligence (AI)
  includes
• design of intelligent machines
• formalization of the notions of intelligence and
  rational behavior
• understanding mechanisms of intelligence
• interaction of humans and intelligent machines.

26
Objectives of AI

• Engineering costruct intelligent machines
• Scientific understand what is intelligence.

27
Can a robot do these?

• Understand?
• Simulate its environment?
• Act rationally?
• Collaborate and compete?
• Display emotions?

A bold claim
A team of Robots will beat the FIFA World Cup
champions by 2050!
28
RoboCup - Aim

• pushing the state-of-the-art
• By mid-21st century, a team of fully autonomous
  humanoid robot soccer players shall win the
  soccer game, comply with the official rule of the
  FIFA, against the winner of the most recent World
  Cup.
• TO BOLDLY GO WHERE MAN HAS GONE BEFORE (cf. Star
  Trek)
• Formalised Testbed

29
Do you really believe that a team of Robots
could beat the FIFA World Cup champions by 2050?

• By all accounts this may sound overly ambitious.
• In fact, if you compare this goal to other ground
  breaking achievements it is not ambitious at
  all.
• The Wright brothers' first airplane was launched
  and 50 years later man landed on the moon.
• Even more recently Deep Blue the computer
  programmed to play chess, played chess grand
  master Garry Kasparov and won -- roughly 50 years
  after the deployment of the first computer.
• It's a long time.
• Think what has happened since 1950.

30
Power of AI
Is the following AI?

• In 1997 a computer, Deep Blue, won a chess match
  with world champion Kasparov.
• Accident?
• IBM paid Kasparov to loose?
• Brute force with no intelligence?
• So, what is intelligence?

31
Simulation

• Turing test (1950)

32
Chess versus soccer robot

• Difference of domain characteristics between
  computer chess and soccer robots

33
Intelligent Agents

• Agents are situated
• Perception of environment
• Execution of actions
• Agents can communicate and collaborate
• they can differ
• than can compete and be more or less
  egoistic/altruistic
• The agents have
• objectives,
• communications,
• intentions.

34
Professor Kim from KAIST
A New Approach
The founder of Robot Soccer and FIRA president
Two organizations 1. FIRA (earlier) 2. RoboCup
(larger)
35
Four Blocks in two PCBs (Printed Circuit Boards)

• Micro-controller (upper PCB)
• Communication module (upper PCB)
• Motor and driving circuits (lower PCB)
• Power (lower PCB)

side view
front view
top view
36
Importance of Robot Soccer

• Communication
• Cooperation
• Coordination
• Learning
• Competence
• Real Time

• Robot Soccer Evolution
• Computer simulations
• Wheeled brainless robots
• Wheeled autonomous robots
• Legged autonomous robots

37
Robot Soccer Purpose

• The number one goal of robot soccer is not
  winning or losing, but accumulating diverse
  technology.
• - Mr. Dao (Senior VP of Sony Corporation).

38
MiroSot
FIRAcategory

• 3 robots on 1 team
• Size 7.5cm 7.5cm 7.5cm
• Ball orange golf ball
• Playground black wooden rectangular
  playground
• (150cm 130cm 5cm)
• Vision global vision system
• (more than 2m above playground)

39
Experimental Setupof the Vision System

• Control panel

40
NaroSot
FIRAcategory

• 5 robots on 1 team
• Size 4cm 4cm 5.5cm
• Ball orange table-tennis ball
• Playground , Vision same as Mirosot

41
KheperaSot
FIRAcategory

• 3 robots on 1 team
• Ball yellow tennis ball
• Playground green playground (105cm 68cm
  20cm)
• Robot Khepera Robot
• Vision K213 Vision Turret

42
RoboSot
FIRAcategory

• 3 robots on 1 team
• Size 15cm 15cm 30cm
• Ball red roller-hockey ball
• Playground black wooden rectangular
  playground (220cm 150cm 30cm)
• Vision on the robot
• Under preparation

43
Small-Size League
44
Small-Size League (F-180)

• Field 2.7 m x 1.5 m

Size Area 18cm rule (fit inside in 18cm
diameter cylinder) Height 15cm (global vision),
22.5cm (otherwise)

• teams of autonomous small size robot play soccer
  game on a field equivalent to a ping-pong table.
• Each team consists of 5 robots.

45
Small size league
The field is the size and color of a Ping Pong
table
46
orange golf ball

• Robots move at speeds as high as 2 meters/second
• Global vision is allowed

47
Robot Soccer Initiative
Vision system
Host computer
Host computer
Communication System
Communication System
Robots on the playing field
Brainless System

• Basic Architecture for Robot Soccer Systems

48
(No Transcript)
49
Vision System

• Vision global vision system(more than 3m above
  ground)

Each team has its own camera and PC
50
Small-Size League

• 20 minutes, 2 breaks

51
Real Robot Small-Size League Competition
52
Middle-Size League
53
Middle-size Real Robot League (F-2000) Local
VISION

• The field is the size and color of a 3 x 3
  arrangement of Ping Pong tables (9-3 5-meter
  field)
• Each team consists of 5 robots playing with a
  Futsal-4 ball (4 players, one goal-keeper)
• Larger (50 centimeters in diameter) robots
• Global vision is not allowed.
• Each robot has its own vision system
• Goals are colored
• Field is surrounded by walls to allow for
  distributed localization through robot sensing
• Rule structure based on the official FIFA rules

54
Medium size league

• Teams of autonomous mid size robots

55
Real Robot Middle-Size League Competition
Ball red small soccer ball (FIFA standard size
4 or 5) Playground green playground (10m 7m
0.5m)
56
Medium Size League
57
Medium Size League
58
(No Transcript)
59
(No Transcript)
60
Robots can be heterogenous
61
Middle-Size League
62
Sony Legged Robot League
63
Sony Legged Robot League

• 3 robots on 1 team (including the goalkeeper).
• Robot AIBO ERS-110 (provided by Sony)

64
No communication, autonomous robots, software
only. Legged Robot League. 2.8 m x 1.8 m2
players and 1 goal-keeper in a team
65
Sony Legged Robot League

• Is played on a field, approx 3x2 meter
• Sony develops the robots, and provides a
  interface for the programming of the robots.

66

• No Hardware modification is allowed

Playing time is 10 minutes per half, with a 10
minute break at halftime
67

• Do different Robots have different personalities?
  
• Some teams have robots with very different
  capabilities.
• But it is hard to think of them as having
  personalities
• rather the robots have different playing styles.
  

68
(No Transcript)
69
Early Sony prototype
70

• Robot movements closely mirror those of animals

71

• The winner is the team that scores the most
  goals.
• In the event of a tie, a sudden death penalty
  kick competition will determine the winner

72
(No Transcript)
73
The Legged Robot League
74
The Legged Robot League
If opposing teams' robots are damaged or play is
excessively rough (whether intentional or not),
penalties may be assessed to the offending robot
75
Humanoid League
76
Starting 2002, the humanoid league
77
Humanoid League

• Bi-Ped League (Humanoid)
• Australia
• Japan

78
(No Transcript)
79
Where is the science in these robot competitions?

• Global vision
• Local vision
• Other sensors
• Cooperation
• Sensor fusion
• Strategy
• Learning

80
Sensors and Actuators for Robot Soccer
Local and Global VISION
81
Sensors for Robot Soccer

• Shaft Encoders
• PI controller to maintain wheel speed
• PI controller to maintain path curvature
• Dead reckoning for vehicle position
  orientation
• Infrared Distance Measurement
• Avoid Collision
• Navigate and map unknown environment
• Update internal position in known environment
• Compass
• Update orientation independent of shaft
  encoders
• Fault-tolerance in case robot gets pushed or
  wheels slip

82
Sensors for Robot Soccer

• Digital Camera
• Low resolution, 60x80 pixels, 24bit color
  (Braunl)
• Color or shape recognition
• Communication
• Sharing information among robots
• Receiving commands from human operator

83
VISION Color Detection

• In robot soccer, objects are color coded
• ball,
• goals,
• opponents,
• team mates,
• walls, etc.
• Teach ball and goal color (hue) before starting
  the game
• Match colors in HSI space
• ? Better in changing lighting conditions

84
Brain-on-board system
Role of Vision

• Robots
• The robots have functions such as velocity
  control, position control, obstacle avoidance,
  etc.
• Host computer
• The host computer processes vision data and
  calculates next behaviors of robots according to
  strategies and sends commands to the robots using
  RF modem.

85
2.2 Robot-based system

• Distributed system
• Intelligent part is implemented in the robots.

• Suitable when the large number of agents exist
• Complex and expensive
• Need communication among robots

86
Robot-based system
Role of vision

• Robots
• The robots decide their own behavior autonomously
  using the received vision data, own sensor data
  and strategies.
• Host computer
• The host computer processes only vision data
• can be considered as a kind of sensor.

87
System Comparisons
Merits
Demerits
Research purpose

• Vision system
• Multi-agent theory

• Cannot use local sensors
• High computing power fast sampling time

Remote-brainless system

• Low cost
• Easy to develop

• Robot system
• Multi-agent system development

Robot -based system

• Complex and expensive robots.
• Hard to build the system

• Suitable for many agents
• Can use local information

• Robot-based and vision-based systems

• Brain-on-board system

• Suitable to modularize

• Risk of inconsistent property between host
  computer and robot system

88
VXD role of color

• Initialization
• Click Load VXD in the Initialize group box
• Click Start Grab
• Configuration
• Load Conf. load a configuration file
• Save Conf. save current configuration
• Set Robot Size set the robot size in number of
  pixels
• Set Pixel Size set the size of each color
  (ball, team,
• robot, opponent)
  patch in number of pixels
• Set Boundary set the field boundary on the
  screen
• Change Color change the color setting of each
  color patch
• Set Color set the range of tolerance of each
  color

89
Subsystems and Vision

• Serial Port
• Select the serial communication port
• Home Goal
• Select the home side on the screen
• Find Objects
• Check the box of which you like to find on the
  field
• Initial Position tell the vision system the
  initial position
• of each object
• E.g.) for the ball
• i) turn on the radio button of Ball
• ii) place the mouse on the ball and press the
  left button
• Repeat above procedure for another object

90
Commands for Vision

• Select Situation
• The situation in which the game is about to start
• Command
• Click Ready the vision system starts finding
  the objects
• on the field
• Click Start the vision system starts sending
  commands
• to the robots
• Click Stop the vision system stops finding
  objects
• and sending commands

91
Tasks for us

• How to organize the ceiling camera system?
• How to describe (learn?) the shapes of robots?
• How to find the (x,y,?) for each robot?
• How to modify the scripted behavior when the
  triplets for each robot are known?
• How to design the interactive behaviors?

92
Task for Robot Theatre Team
For next week

• Write a half-page essay about the vision system
  that we discussed today.
• Use you knowledge from other lecture of today.
• Add your imagination and crazy ideas about what
  the robots should see and know for our particular
  scene of Marie Curie and Schroedingers Cat.