Title: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision
1Research in Intelligent Mobile Robotics (and
related topics)Part 1 Navigation and Vision
- Anna Helena Reali Costa
- anna.reali_at_poli.usp.br www.pcs.usp.br/anna
- Laboratório de Técnicas Inteligentes
- Escola Politécnica da Universidade de São Paulo
- Carlos Henrique Costa Ribeiro
- carlos_at_comp.ita.br www.comp.ita.br/carlos
- Divisão de Ciência da Computação
- Instituto Tecnológico de Aeronáutica
2Preface
- This is a two-part talk about the research on
Intelligent Mobile Robotics and related topics
at - LTI USP (Laboratório de Técnicas Inteligentes
Universidade de São Paulo, Brazil) - NCROMA-ITA (Laboratório de Navegação e Controle
de Robôs Móveis Autônomos Instituto Tecnológico
de Aeronáutica, Brazil). - These research groups are involved in the
MultiBot cooperation project CAPES/GRICES with
ISLab-IST.
3LTI - EPUSP
- Prof. Anna Reali
- 5 PhD Students
- Alexandre Simões, Reinaldo Bianchi, Valdinei
Silva, Valguima Odakura, Waldemar Bonventi. - 3 Master Students
- Alexandre Cunha, Antônio Selvatici, Luiz Carlos
Maia - 3 Undergraduate Students
- Rafael Pacheco, Márcio Seixas, Júlio Kawai
- 2 Final Course Projects
4NCROMA ITA
- Prof. Carlos Ribeiro
- 1 PhD Student
- Letícia Friske
- 5 Master Students
- Luís Almeida, Ricardo Maia, Juliano Pereira,
- Esther Colombini, Celeny Alves
- 2 Undergraduate Students
- Lucas Gabrielli, Fábio Miranda
5Contents
Intelligent Mobile Robotics
- Part 1
- Navigation
- Map building
- Localization
- Perception
- Computer Vision
- Part II
- Learning
6Map Building
- Our goal Test map building algorithms in real
robots - fast enough?
- precise enough?
- ok for learning applications (e.g. path learning)?
7Map BuildingSildomar Takahashi, Carlos
RibeiroRoberto Barra, Ricardo Domenecci, Anna
Reali
- Efficient Learning of Variable-Resolution
Cognitive Maps for Autonomous Indoor Navigation.
- Arleo, Millan e Floreano, IEEE-SMC, 1999.
- Advantages
- Complete algorithmic description
- Simple structure
- Limitations
- Assumes structure (orthogonal obstacles / walls)
- Reliance on dead-reckoning (but can be adapted to
more sophisticated localization)
8The Basic Algorithm
- Explores environment
- Once an obstacle is detected
- Determines obstacle frontiers either via
- An a priori sensor model
- A pre-trained neural net
- Includes obstacle in the global map
- Defines new partition to explore. If there is
none, END. Else finds route to new partition. - Executes route and explores new partition. Once
an obstacle is detected, Step 2. Else, Step 1.
9Detection of obstacle frontiers
- Either a priori model or neural net model
- Integration over time
- Straight-line adjustment and correction
(according to a priori actuator model)
10Very Scientific Set-up
3 x 3,5 m
11Global Maps Magellan, Neural net model
Map 1
Map 2
Map 3
12Global Map Pioneer, a priori model
straight-line model-based correction
13Conclusions
- Tested algorithm (possibly with some
modifications) is a good compromise
efficiency/precision for realistic applications
fast yet fairly accurate. - Next steps
- Studies on simultaneous localization and mapping
(SLAM algorithms). - Valguima Odakura (Anna Reali) SLAM based on
visual landmarks. - Fabio Miranda (Carlos Ribeiro) Bayesian landmark
learning. - Techniques for map building acceleration.
14Markov LocalizationLuís Almeida, Carlos
RibeiroJúlio Kawai, Anna Reali
- Position estimation based on Bayesian update
- Belief update based on sensor info
- Belief update based on action info
- Sensor/actuator models and initial belief
distribution arbitrary. - Simple to implement.
- Computationally costly (Monte Carlo
implementation particle filters is a possible
fix).
15Markov Localization
16Monte Carlo Localization GA OptimizationLuís
Almeida, Carlos Ribeiro
GA on population of particles (fitness as
combination of belief / particle cluster
distribution)
GA on population of particles (fitness as
combination of belief / particle cluster
distribution)
MC
GA
MC
MC
GA
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
- Basic idea use GA to create a better set of
particles for next MC update. - Initial results ok (in need of statistical
validation).
17Next Steps
- Validation of GA approach.
- Better sensor and actuator models.
- Implementation in a real robot.
- Literature on Monte Carlo methods (applications
on signal detection and tracking) many
variations to be tried...
18Computational Vision
- Image Segmentation
- Using Color Classification
- Using Background Model
- Using Optical Flow
- Based on Binocular Stereo Vision
19Color Classification - I
- Using threshold values
- In the color representation space
- Neural Network MLP backpropagation alg.
- Alexandre Simões, Anna Reali
20Derived ApplicationAlexandre Simões, Anna Reali
Orange Classifier - CEAGESP, SP
21Non-supervised iterative fuzzy color
classification Waldemar Bonventi, Anna Reali
- For number of clusters 2 to Cmax, do
- Apply FCM-GK (non-supervised fuzzy classifier) to
RGB image - Calculate the ratio c/s for each cluster set
- c Cluster dimension/number of members
- s Separation among clusters
- Choose the cluster set, based on c/s.
- Show color classification result for the best
cluster set.
22An example soccer
The best cluster set ? 6 clusters
23Another example Rio de Janeiro
The best cluster set ? 3 clusters
24Computational Vision
- Image Segmentation
- Using Color Classification
- Using Background Model
- Using Optical Flow
- Based on Binocular Stereo Vision
25Background Model - I
- Background subtraction
- Thresholding the error between an estimate of the
image without moving objects M(C) and the
current image - ? Model can not adapt to environment changes!
26Background Model IIMárcio Seixas, Anna Reali
- Time-Adaptive, Per-Pixel Mixture-of-Gaussians
- Time series of observations at a given pixel (its
color) is modeled by a mixture-of-gaussians. - Based on the persistence and the variance of each
of the gaussians of the mixture, it is determined
which gaussians may correspond to background
colors. - Hypothesis gaussian distributions with low
variance and high persistence correspond to
background model. - Per-pixel models are updated as new observations
are obtained (according to a learning rate). - ? It is capable of dealing with long-term scene
changes (e.g. lighting changes)!
27Derived Applicationplatform occupancy
Terminal Rodoviário de Santo Amaro TRENDS
Prefeitura de São Paulo Márcio Seixas, Anna Reali
Fixed model M(C)
28Computational Vision
- Image Segmentation
- Using Color Classification
- Using Background Model
- Using Optical Flow
- Based on Binocular Stereo Vision
29Optical Flow - idea
30Vision-based robotic behaviorAntonio Selvatici,
Anna Reali
- Robot (with camera) navigating in a stationary
scenario. - Calculation of the optical-flow divergent to
estimate the time-to-crash value in order to
avoid collisions with obstacles. - We are now investigating a robust method to
directly calculate the per-pixel time-to-crash
value
Original sequence
Pixel time-to-crash
Filtered values
Gray levels near ?bright far ? dark Black
unknown distance
31Derived Application monitoring of underground
rail tracksLuiz Maia, Anna Reali
ALSTOM Metrô de São Paulo
32Computational Vision
- Image Segmentation
- Using Color Classification
- Using Background Model
- Using Optical Flow
- Based on Binocular Stereo Vision
33Binocular Stereo VisionRafael Pacheco, Anna Reali
- Distance-Map Calculation
- Calibration Zhang, ICCV 99
- Matching blob coloringcentroidcorrelation
- Triangulation
- Segmentation based on color distance-map.
34Derived ApplicationOutdoors Measurement
TRENDS Prefeitura de São Paulo Rafael Pacheco,
A. Reali
35Conclusions
- In CV, we are now investigating
- Automatic learning of fuzzy color classifiers
Waldemar Bonventi, LTI - A framework for high-level feedback to adaptive,
per-pixel, mixture-of-gaussian background models
Márcio Seixas, LTI - Mathematical formulation for direct and robustly
calculate the per-pixel, time-to-crash values,
considering a moving observer in a stationary
scenario Antonio Selvatici, LTI - Distributed, real-time approach to calculate the
optical flow, considering a stationary observer
in a dynamic scenario Luiz Maia, LTI.