Roboteyepair an ultra-high speed stereo vision system based on CNN technology

1
Roboteyepairan ultra-high speed stereo vision
system based on CNN technology
2
Motivation

• The mechanical properties of robots enable faster
  operation than their current vision systems
• A high-speed, medium resolution, low cost vision
  system has many potential application fields in
  the industry
• Both stereo and mono versions have potential
  applications

3
Goals

• Main characteristics of the Roboteyepair
• Standalone vision system
• Capable to capture and process image pairs at
  frame rates up to 5000 per second
• Medium resolution
• High performance focal plane CNN processors
• Equipped with serial and Ethernet communication
  ports
• Developed for industrial use.

4
The CNN visual processor

• 128x128 sensor-processor array
• Optimized for performing array operations
• on analog and logic values
• Time to perform a typical operation
• Logic 300ns (morphology, binary
• logic operations, skeletonizing, etc.)
• Analog 3-5ms (diffusion, edge detection, Sobel,
  etc..)
• 8 internal image memories
• 128Mpixel/sec I/O speed (128 internal AD/DA)
• 0.35mm technology, 12x12mm silicon surface
• Operates as a coprocessor, a master processor is
  needed

5
Block diagram of the system
6
Realization of the system
7
The finished boards
8
The complete system
9
Programming the DSP

• Image processing
• Native languages (AMC, Alpha, C)
• Handling two CNN chips
• The target chip has to be specified for every
  command
• Synchronization
• Adapting CNN image processing library
• Communication using the ETRAX chip

10
Programming the communication processor

• ETRAX 100 2.4 LINUX kernel
• Driver (ETRAX ??XBUS)
• TCP/IP communication module (INET Daemon)
• Assigns the communication driver to the given TCP
  port
• standard input ?? standard output redirecting
• A special daemon has also been implemented
• Boot device

11
Monitoring software on the PC side

• Why is it needed
• Supporting system setup
• Application development and testing
• Parameter adjustment
• Testing camera parameters
• Capabilities
• Multi-window graphical interface
• Editor, compiler and software downloading devices

12
The project will be finished in May, 2003
13
Applications

• Stereo machine vision system for robots
• Visual navigation for NASA Mars Mission 2007
• Non-invasive speed and displacement measurement,
  quality control of narrow textile bands and tapes
  (textile industry)
• Classification of large number of tiny objects
  (pills, grains, pebbles, etc.)
• Monitoring fast industrial processes (sparkplug)

14
Future developments

• Replace the current DSP with a TMSC3206115 DSP
  running _at_600 MHz
• Equip the system with an optional 1.3 MPixel CMOS
  sensor
• Support of communication via USB port
• These developments are expected to be finished by
  September, 2003

15
Thank you for your attention!