Carolina Galleguillos and Serge Belongie - PowerPoint PPT Presentation

About This Presentation
Title:

Carolina Galleguillos and Serge Belongie

Description:

Grocery shopping is a common activity that people all over the ... Froogle, Shopwiki, Amazon. - Groceries, Yahoo images - General specialized (UPC code) ... – PowerPoint PPT presentation

Number of Views:46
Avg rating:3.0/5.0
Slides: 2
Provided by: groziC
Category:

less

Transcript and Presenter's Notes

Title: Carolina Galleguillos and Serge Belongie


1
GroZi a Grocery Shopping Assistant for the Blind
Carolina Galleguillos and Serge
Belongie Department of Computer Science and
Engineering, UCSD cgallegu,sjb_at_cs.ucsd.edu
Abstract
Object Recognition
Use of the System
  • Two types of recognition (mn, mltltn)
  • Detection (of objects).
  • Verification (objects detected are in that
    list).
  • Algorithms
  • Color histogram matching.
  • SIFT features matching.
  • Haar like features in Adaboost framework.
  • Shared features.

Grocery shopping is a common activity that people
all over the world perform on a regular basis.
Unfortunately, grocery stores and supermarkets
are still largely inaccessible to people with
visual impairments, as they are generally viewed
as "high cost" customers. We propose to develop
a computer vision based grocery shopping
assistant based on a handheld device with haptic
feedback that can detect different products
inside of a store, thereby increasing the
autonomy of blind (or low vision) people to
perform grocery shopping. Our solution makes use
of new computer vision techniques for the task of
visual recognition of specific products inside of
a store as specified in advance on a shopping
list. These techniques can avail of complementary
resources such as RFID, barcode scanning, and
sighted guides. We also present a challenging
new dataset of images consisting of different
categories of grocery products that can be use
for object recognition studies.
Create a Shopping List
  • Online Website
  • Website stores data and images of different
    products.
  • Feedback from users.
  • Provides walking path.
  • Prepare shopping list
  • Download information into Mozi Box.

CbCr Histograms
Get to the Grocery Store
SIFT keypoints
  • Separate project.
  • Mozi Box with GPS.
  • Visual waypoints.
  • Traffic/Street sign reading.
  • Use in addition to cane and asking sighted
    bystanders.

Haar features
CbCr chrominance plane. L2 distance between
histograms is used for matching.
Navigate the Store
Motivations
Integral histogram applied for better performance
Porikli, 2005. Processing time 25 frames per
second.
  • Finding aisle
  • (OCR, RFID, ask).
  • Avoiding obstacles (cane).
  • Finding products (sweep of aisle, spot product,
    barcode check).
  • Checking out (coupon and cash).
  • Increase independence of people with low vision
    (specially blind) to perform grocery shopping in
    a supermarket or store.
  • There are 1.3 million legally blind people in
    the U.S.
  • Help to plan shopping list, walking path to the
    store and grocery shopping.
  • Advance research on object recognition for
    mobile robotics with constrained computing
    resources.

Text detection is also part of the GroZi project.
Obtaining Data
Adaboost and Haar features are used to detect
text on images.
  • Training data was obtained from two major
    sources
  • Sunshine Store _at_ UCSD (video capture)
  • - 30 minutes of capture
  • - Divided into 29 .avi files
  • - Boxes containing product
  • manually cropped
  • Web (online images)
  • - Froogle, Shopwiki, Amazon.
  • - Groceries, Yahoo images
  • - General specialized (UPC code)
  • queries.
  • Test data will be obtained from
  • Collected videos/images from MoZi box (in situ).

MoZi Box
Future Directions
General purpose low-cost mobile system geared for
computer vision applications.
  • Obtain a final version of the data set that can
    be viewed online, which is continuously updated,
    with more products and user interaction.
  • Test detection and recognition of products
    using SIFT on top of Haar like features.
  • Apply color Haar like features (opponent
    channels) for a better detection.
  • Bar code scanning to perform active learning in
    order to learn relations between barcodes and
    product images.
  • Finite memory Compact Flash (CF) cards ranging
    from 256 MB to 4 GB.
  • Processor speed in the neighborhood of
    60-400MHz.
  • Frame rate enough snapshots to cover the shelf
    with some overlap (as in panoramic stitching) .
  • Color Calibration Macbeth color chart to
    calibrate the color space.

Dataset
Acknowledgments
  • Special thanks to Stephan Steinbach from Calit2,
    Michele Merler and Tom Duerig from UCSD computer
    science department.

The GroZi data set corresponds to 128 different
grocery products, that have been obtained from
online images and collected videos.
Web image
captured image
Write a Comment
User Comments (0)
About PowerShow.com