Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision

1
Research in Intelligent Mobile Robotics (and
related topics)Part 1 Navigation and Vision

• Anna Helena Reali Costa
• anna.reali_at_poli.usp.br www.pcs.usp.br/anna
• Laboratório de Técnicas Inteligentes
• Escola Politécnica da Universidade de São Paulo
• Carlos Henrique Costa Ribeiro
• carlos_at_comp.ita.br www.comp.ita.br/carlos
• Divisão de Ciência da Computação
• Instituto Tecnológico de Aeronáutica

2
Preface

• This is a two-part talk about the research on
  Intelligent Mobile Robotics and related topics
  at
• LTI USP (Laboratório de Técnicas Inteligentes
  Universidade de São Paulo, Brazil)
• NCROMA-ITA (Laboratório de Navegação e Controle
  de Robôs Móveis Autônomos Instituto Tecnológico
  de Aeronáutica, Brazil).
• These research groups are involved in the
  MultiBot cooperation project CAPES/GRICES with
  ISLab-IST.

3
LTI - EPUSP

• Prof. Anna Reali
• 5 PhD Students
• Alexandre Simões, Reinaldo Bianchi, Valdinei
  Silva, Valguima Odakura, Waldemar Bonventi.
• 3 Master Students
• Alexandre Cunha, Antônio Selvatici, Luiz Carlos
  Maia
• 3 Undergraduate Students
• Rafael Pacheco, Márcio Seixas, Júlio Kawai
• 2 Final Course Projects

4
NCROMA ITA

• Prof. Carlos Ribeiro
• 1 PhD Student
• Letícia Friske
• 5 Master Students
• Luís Almeida, Ricardo Maia, Juliano Pereira,
• Esther Colombini, Celeny Alves
• 2 Undergraduate Students
• Lucas Gabrielli, Fábio Miranda

5
Contents
Intelligent Mobile Robotics

• Part 1
• Navigation
• Map building
• Localization
• Perception
• Computer Vision
• Part II
• Learning

6
Map Building

• Our goal Test map building algorithms in real
  robots
• fast enough?
• precise enough?
• ok for learning applications (e.g. path learning)?

7
Map BuildingSildomar Takahashi, Carlos
RibeiroRoberto Barra, Ricardo Domenecci, Anna
Reali

• Efficient Learning of Variable-Resolution
  Cognitive Maps for Autonomous Indoor Navigation.
  
• Arleo, Millan e Floreano, IEEE-SMC, 1999.
• Advantages
• Complete algorithmic description
• Simple structure
• Limitations
• Assumes structure (orthogonal obstacles / walls)
• Reliance on dead-reckoning (but can be adapted to
  more sophisticated localization)

8
The Basic Algorithm

• Explores environment
• Once an obstacle is detected
• Determines obstacle frontiers either via
• An a priori sensor model
• A pre-trained neural net
• Includes obstacle in the global map
• Defines new partition to explore. If there is
  none, END. Else finds route to new partition.
• Executes route and explores new partition. Once
  an obstacle is detected, Step 2. Else, Step 1.

9
Detection of obstacle frontiers

• Either a priori model or neural net model
• Integration over time
• Straight-line adjustment and correction
  (according to a priori actuator model)

10
Very Scientific Set-up
3 x 3,5 m
11
Global Maps Magellan, Neural net model
Map 1
Map 2
Map 3
12
Global Map Pioneer, a priori model
straight-line model-based correction
13
Conclusions

• Tested algorithm (possibly with some
  modifications) is a good compromise
  efficiency/precision for realistic applications
  fast yet fairly accurate.
• Next steps
• Studies on simultaneous localization and mapping
  (SLAM algorithms).
• Valguima Odakura (Anna Reali) SLAM based on
  visual landmarks.
• Fabio Miranda (Carlos Ribeiro) Bayesian landmark
  learning.
• Techniques for map building acceleration.

14
Markov LocalizationLuís Almeida, Carlos
RibeiroJúlio Kawai, Anna Reali

• Position estimation based on Bayesian update
• Belief update based on sensor info
• Belief update based on action info
• Sensor/actuator models and initial belief
  distribution arbitrary.
• Simple to implement.
• Computationally costly (Monte Carlo
  implementation particle filters is a possible
  fix).

15
Markov Localization
16
Monte Carlo Localization GA OptimizationLuís
Almeida, Carlos Ribeiro
GA on population of particles (fitness as
combination of belief / particle cluster
distribution)
GA on population of particles (fitness as
combination of belief / particle cluster
distribution)
MC
GA
MC
MC
GA
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)

• Basic idea use GA to create a better set of
  particles for next MC update.
• Initial results ok (in need of statistical
  validation).

17
Next Steps

• Validation of GA approach.
• Better sensor and actuator models.
• Implementation in a real robot.
• Literature on Monte Carlo methods (applications
  on signal detection and tracking) many
  variations to be tried...

18
Computational Vision

• Image Segmentation
• Using Color Classification
• Using Background Model
• Using Optical Flow
• Based on Binocular Stereo Vision

19
Color Classification - I

• Using threshold values
• In the color representation space
• Neural Network MLP backpropagation alg.
• Alexandre Simões, Anna Reali

20
Derived ApplicationAlexandre Simões, Anna Reali
Orange Classifier - CEAGESP, SP
21
Non-supervised iterative fuzzy color
classification Waldemar Bonventi, Anna Reali

• For number of clusters 2 to Cmax, do
• Apply FCM-GK (non-supervised fuzzy classifier) to
  RGB image
• Calculate the ratio c/s for each cluster set
• c Cluster dimension/number of members
• s Separation among clusters
• Choose the cluster set, based on c/s.
• Show color classification result for the best
  cluster set.

22
An example soccer
The best cluster set ? 6 clusters
23
Another example Rio de Janeiro
The best cluster set ? 3 clusters
24
Computational Vision

• Image Segmentation
• Using Color Classification
• Using Background Model
• Using Optical Flow
• Based on Binocular Stereo Vision

25
Background Model - I

• Background subtraction
• Thresholding the error between an estimate of the
  image without moving objects M(C) and the
  current image
• ? Model can not adapt to environment changes!

26
Background Model IIMárcio Seixas, Anna Reali

• Time-Adaptive, Per-Pixel Mixture-of-Gaussians
• Time series of observations at a given pixel (its
  color) is modeled by a mixture-of-gaussians.
• Based on the persistence and the variance of each
  of the gaussians of the mixture, it is determined
  which gaussians may correspond to background
  colors.
• Hypothesis gaussian distributions with low
  variance and high persistence correspond to
  background model.
• Per-pixel models are updated as new observations
  are obtained (according to a learning rate).
• ? It is capable of dealing with long-term scene
  changes (e.g. lighting changes)!

27
Derived Applicationplatform occupancy
Terminal Rodoviário de Santo Amaro TRENDS
Prefeitura de São Paulo Márcio Seixas, Anna Reali
Fixed model M(C)
28
Computational Vision

• Image Segmentation
• Using Color Classification
• Using Background Model
• Using Optical Flow
• Based on Binocular Stereo Vision

29
Optical Flow - idea
30
Vision-based robotic behaviorAntonio Selvatici,
Anna Reali

• Robot (with camera) navigating in a stationary
  scenario.
• Calculation of the optical-flow divergent to
  estimate the time-to-crash value in order to
  avoid collisions with obstacles.
• We are now investigating a robust method to
  directly calculate the per-pixel time-to-crash
  value

Original sequence
Pixel time-to-crash
Filtered values
Gray levels near ?bright far ? dark Black
unknown distance
31
Derived Application monitoring of underground
rail tracksLuiz Maia, Anna Reali
ALSTOM Metrô de São Paulo
32
Computational Vision

• Image Segmentation
• Using Color Classification
• Using Background Model
• Using Optical Flow
• Based on Binocular Stereo Vision

33
Binocular Stereo VisionRafael Pacheco, Anna Reali

• Distance-Map Calculation
• Calibration Zhang, ICCV 99
• Matching blob coloringcentroidcorrelation
• Triangulation
• Segmentation based on color distance-map.

34
Derived ApplicationOutdoors Measurement
TRENDS Prefeitura de São Paulo Rafael Pacheco,
A. Reali
35
Conclusions

• In CV, we are now investigating
• Automatic learning of fuzzy color classifiers
  Waldemar Bonventi, LTI
• A framework for high-level feedback to adaptive,
  per-pixel, mixture-of-gaussian background models
  Márcio Seixas, LTI
• Mathematical formulation for direct and robustly
  calculate the per-pixel, time-to-crash values,
  considering a moving observer in a stationary
  scenario Antonio Selvatici, LTI
• Distributed, real-time approach to calculate the
  optical flow, considering a stationary observer
  in a dynamic scenario Luiz Maia, LTI.