An appearance-based visual compass for mobile robots

1
An appearance-based visual compass for mobile
robots

• Jürgen Sturm

University of Amsterdam Informatics Institute
2
Overview

• Introduction to Mobile Robotics
• Background (RoboCup, Dutch Aibo Team)
• Approach
• Results
• Conclusions

2
3
Mobile robots
Robot cranes and trucks unloading ships Port of
Rotterdam
Sony Aibos playing soccer Cinekids, De Balie,
Amsterdam
SICO at Kosair Children's Hospital Dometic,
Louisville, Kentucky
RC3000, the robocleaner Kärcher
4
Challenge Application Robot Soccer
4
5
Robot localization

• Robot localization
• .. is the problem of estimating the robots pose
  relative to a map of the environment.
• Probabilistic approaches
• Noise
• Ambiguity
• Uncertainty

5
6
Design

• Sensors
• Wheelsensors, GPS, Laserscanner, Camera..
• Feature space
• Map and Belief Representation
• Grid-based Maps, Topological graphs
• Single/Multi Hypothesis Trackers
• Filters
• Kalman Filter, Monte-Carlo Methods

6
7
Design of Classical Approaches

• Artificial environments
• (Electro-magnetic) guiding lines
• (Visual) landmarks
• Special sensors
• GPS
• Laser-range-scanners
• Omni-directional cameras
• Computationally heavy
• offline computation

7
8
Design of New Approach

• Natural environments
• Human environments
• Unstructured and unknown for the robot
• Normal sensors
• Camera
• Reasonable requirements
• Real-time
• On-board

8
9
Platform Sony Aibo

• Computer
• 64bit RISC processor
• 567 MHz
• 64 MB RAM
• 16 MB memorystick
• WLAN

• Internal camera
• 30fps
• 208x160 pixels

• Actuators
• Legs 4 x 3 joints
• Head 3 joints

9
10
Approach
10
11
Demo VideoVisual Compass
11
12
Approach - Synopsis
12
13
Previously learned map
Sector
-
based
Raw image
Color class image
feature extraction
Image data
Motion Model
Sensor Model
Motion data
Estimated Motion
Correlation Likelihoods
Localization Filter
13
prior
odometry-corrected
posterior
14
Sector-based feature extraction (1)
Camera field of view 50 Head field of view 230
14
15
Sector-based feature extraction (2)

• For each sector
• Count color class transitions in vertical
  direction
• Compute relative transition frequencies

15
16
Sensor model (1)

• Relative frequency of transitions from color
  class i to color class j in direction f
• Frequency measurements originate from a
  probabilistic source (distribution)
• How to approximate these distributions?

16
17
Sensor model (2)

• Approximate source by a histogram distribution
• (parameters constitute the map)

17
18
Sensor model (2)

• Likelihood that a single frequency measurement
  originated from direction f
• Likelihood that a full feature vector (one
  sector) originated from direction f
• Likelihood that a camera image (set of features)
  originated from direction f

18
19
Sensor model (2)

• Likelihood that a single frequency measurement
  originated from direction f
• Likelihood that all frequency measurements
  originated from direction f
• Likelihood that whole camera image originated
  from direction f

20
Localization filterOrientational component

• Use a Bayesian Filter to update robots beliefs
  (circular grid buffer)
• From this buffer, extract per time step
• Heading estimate
• Variance estimate

prior
odometry-corrected
posterior
20
21
Results
22
Results Brightly illuminated living room

• Applicable in natural indoor environment
• Good accuracy (error lt5)

22
23
Results Daylight office environment

• Applicable in natural office environment
• Very robust against displacement
• (error lt20 over 15m)

23
24
Results Outdoor soccer field

• Applicable in natural outdoor environment

24
25
Results RoboLab 4-Legged soccer field

• Applicable in RoboCup soccer environment

25
26
Results RoboLab 4-Legged soccer field

• True average error lt10 on a grid of 3x3m

26
27
Results Variable and Parameter Studies

• Distance to training spot
• Changes in illumination
• Angular resolution
• Scanning grid coverage
• Number of color classes

27
28
Localization filterTranslational component

• Use multiple training spots
• Each (projectively distorted) patch yields
  slightly different likelihoods
• Interpolate translation from these likelihoods
• Visual Homing

28
29
Demo VideoVisual Homing
29
30
Results Visual Homing
Robot walks back to center after kidnap
100
75
Positioning accuracy
50
25
0
y cm
-100
-75
-50
-25
0
25
50
75
100
-25
-50
-75
-100
x cm

• Proof of concept

30
31
Conclusions

• Novel approach to localization
• Works in unstructured environments
• Accurate, robust, efficient, scaleable
• Interesting approach for mobile robots

32
Future Research

• Use Monte-Carlo Localization
• Extend to dynamic environments
• Triangulation from two training spots
• Announced succeeding projects
• Port to RoboCup Rescue Simulation (MSc. Project)
• RoboCup 2007 Open Challenge (DOAS Project)

32
33
3rd Prize Technical Challengesof the 4-Legged
League, RoboCup 2006 in Bremen
33
34

• Thank You!