Textual Information Access for the Visually Impaired

1
Textual Information Access for the Visually
Impaired

• Ramani Duraiswami

2
Project

• Approximately 1 million blind and 5 million
  visually impaired people in the US
• Comparable fractions of the population elsewhere
• Their Goal Lead an independent productive life
• To do this they need to understand a significant
  amount of textual information from the environment

• Examples
• Signs on streets, supermarkets/groceries,
  buildings
• Labels on medicines, products
• Instructions on equipment, computers
• Newspapers, magazines
• Our goal Provide them with devices to read text
• Computer vision, Pattern recognition (OCR),
  computational audio, and wearable computers to
  help them.

3
Restricted initial goal magazine reader

• Tool targeted to people with some vision
• Use optical character recognition (OCR) software
• OCR works with flat, high resolution scanned
  text.
• Read out the recognized text with text-to-speech
  (TTS) software
• Goal read text with a video camera, OCR
  software, portable/wearable computer, headphones

• Problems
• Text is on a surface that is mostly planar or
  curved
• Warped by perspective projection to the camera
• Camera images are low resolution (640 x 480)
• OCR uses scanned images at 600 to 2400 dpi

4
System Design

• For good OCR
• Characters must be 30-60 pixels tall
• Characters must not be skewed
• Camera captures initial image of page
• Using initial images
• Estimate font size and page structure
• Determine scanning pattern
• Zoom into pieces of the image
• Register each zoomed piece with the big picture
• Capture image at optimum focus and zoom
• To make sure pieces are not too small
• Use super-resolution techniques
• Mosaic pieces together and dewarp

5
Super-resolution

• Convert a sequence of many (different) coarsely
  sampled images into a high resolution image.