Image Understanding - PowerPoint PPT Presentation

Loading...

PPT – Image Understanding PowerPoint presentation | free to download - id: 26c428-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Image Understanding

Description:

High-level: extract meaningful information from the low-level representation. ... Two regions that have identical spectra result in different color (lightness) ... – PowerPoint PPT presentation

Number of Views:102
Avg rating:3.0/5.0
Slides: 60
Provided by: jjg1
Learn more at: http://www.cs.rit.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Image Understanding


1
Image Understanding
  • Roxanne Canosa, Ph.D.

2
Introduction
  • Computer vision
  • Give machines the ability to see
  • The goal is to duplicate the effect of human
    visual processing
  • We live in a 3-D world, but camera sensors can
    only capture 2-D information.
  • Flip side of computer graphics?

3
Introduction
  • Computer vision is composed of
  • Image processing
  • Image analysis
  • Image understanding

4
Introduction
  • Image processing
  • The goal is to present the image to the system in
    a useful form
  • image capture and early processing
  • remove noise
  • detect luminance differences
  • detect edges
  • enhance image

5
Introduction
  • Image analysis
  • The goal is to extract useful information from
    the processed image
  • identify boundaries
  • find connected components
  • label regions
  • segment parts of objects
  • group parts together into whole objects

6
Introduction
  • Image understanding
  • The goal is to make sense of the information.
    Draw qualitative, or semantic, conclusions from
    the quantitative information.
  • make a decision about the quantitative
    information
  • classify the parts
  • recognize objects
  • understand the objects usage and the meaning of
    the scene

7
Introduction
  • Computer vision uses techniques and methods from
  • electronics - sensor technology
  • mathematics - statistics and differential
    calculus
  • spatial pattern recognition
  • artificial intelligence
  • psychophysics

8
Low-level Representations
  • Low-level little knowledge about the content of
    the image
  • The data that is manipulated usually resembles
    the input image. For example, if the image is
    captured using a CCD camera (2-D), the
    representation can be described by an image
    function whose value is brightness depending on 2
    parameters the x-y coordinates of the location
    of the brightness value.

9
Low-Level Mechanisms
  • Low-level vision only takes us to the
    sophistication of a very expensive digital camera

10
High-level Representations
  • High-level extract meaningful information from
    the low-level representation.
  • Image may be mapped to a formalized model of the
    world (model may change dynamically as new
    information becomes available)
  • Data to be processed is dramatically reduced
    instead of dealing with pixel values, deal with
    features such as shape, size, relationships, etc
  • Usually expressed in symbolic form

11
High-Level Mechanisms
  • High-level vision and perception requires brain
    functions that we do not fully understand yet

12
Bottom-up vs. Top-down
  • Bottom-up processing is content-driven
  • Top-down processing is context-driven
  • Goal combine knowledge about content as well as
    context.
  • Goals, plans, history, expectations
  • Imitate human cognition and the ability to make
    decisions based on extracted information

13
Bottom-up v. Top-down
Top-Down?
Bottom-up?
Information flow
Information flow
14
Top-down Control
Visual Completion
15
Top-down Control
Visual Completion
16
Top-down Control
Visual Completion
17
Top-down Control
Visual Completion
18
Top-down Control
Visual Completion
19
Old Women or Young Girl?
http//dragon.uml.edu/psych/woman.html
20
Expectation and Learning
From Palmer (1999)
21
Zolner Illusion
Are the black and yellow lines parallel?
http//www.torinfo.com/illusion/illus-17.html
22
Visual Illusions Demos
http//www.michaelbach.de/ot/index.html
23
The Human Visual System
  • Optical information from the eyes is transmitted
    to the primary visual cortex in the occipital
    lobe at the back of the head.

24
The Human Visual System
- 20 mm focal length lens - iris controls amount
of light entering eye by changing the size of
the pupil
25
The Human Visual System
  • Light enters the eye through the cornea, aqueous
    humor, lens, and vitreous humor before striking
    the light-sensitive receptors of the retina.
  • After striking the retina, light is converted
    into electrochemical signals that are carried to
    the brain via the optic nerve.

26
The Human Visual System
image from www.photo.net/photo/edscott/vis00010.ht
m
27
Multi-Resolution Vision

28
Multi-Resolution Vision
If you can read this you must be cheating
29
Multi-Resolution Vision
  • The distribution of rods and cones across the
    retina is highly uneven
  • The fovea contains the highest concentration of
    cones for high visual acuity

From Palmer (1999)
30
Contrast Sensitivity
31
Lateral Inhibition
32
Lateral Inhibition
33
Lateral Inhibition
  • A biological neural network in which neurons
    inhibit spatially neighboring neurons.
    Architecture of first few layers of retina.

10
10
5
5
5
10
5
10
Input light level
-0.2
-0.2
-0.2
-0.2
-0.2
-0.2
-0.2
Layer n
1
1
1
1
1
1
Layer n 1
10-2-2
10-2-2
10-2-1
5-2-1
5-1-1
5-1-1
34
Simultaneous Contrast
  • Two regions that have identical spectra result in
    different color (lightness) perceptions due to
    the spectra of the surrounding regions
  • Background color can visibly affect the perceived
    color of the target

35
Simultaneous Contrast
36
Task-Oriented Vision
Judge their ages
Free Viewing
Estimate the economic level of the people
Remember the clothes worn by the people
Guess what they had been doing before the
visitors arrival
37
Change Blindness
  • Lack of attention to an object causes failure to
    perceive it
  • People find it difficult to detect major changes
    in a scene if those changes occur in objects that
    are not the focus of attention
  • Our impression that our visual capabilities give
    us a rich, complete, and detailed representation
    of the world around us is a grand illusion!

38
Change Blindness Demos
http//www.usd.edu/psyc301/ChangeBlindness.htm
http//viscog.beckman.uiuc.edu/djs_lab/demos.html
39
Modeling Attention
  • How do we decide where to look next while
    performing a task?
  • What factors influence our decision to look at
    something?
  • Can we model visual behavior?

40
Modeling Attention - Saliency Maps
  • Koch Ullman (1985), Itti Koch (2000),
    Parkhurst, Law, Neibur (2002), Turano,
    Geruschat, Baker (2003)

41

Computational Model of Saliency
color
intensity
orient
Saliency Map
center surrounds
42
XYZ transform
Input Image (RGB)
L M S
Pre-processing Module
Color Map
A C1 C2
Intensity Map
Oriented Edge Module
Object Module
G45
G90
G135
G0
Orientation Map
Proto-object Map
Conspicuity Map
43
XYZ transform
Input Image (RGB)
L M S
Color Map
A C1 C2
Intensity Map
Oriented Edge Module
Object Module
G45
G90
G135
G0
Orientation Map
Proto-object Map
Conspicuity Map
44
XYZ transform
Input Image (RGB)
L M S
Pre-processing Module
Color Map
A C1 C2
Intensity Map
Object Module
G45
G90
G135
G0
Orientation Map
Proto-object Map
Conspicuity Map
45
XYZ transform
Input Image (RGB)
L M S
Pre-processing Module
A C1 C2
Intensity Map
Oriented Edge Module
G45
G90
G135
G0
Orientation Map
Proto-object Map
Conspicuity Map
46
Weight Output with Contrast Sensitivity Function
CSF 2.6 (0.0192 0.114f) e -(o.114f) 1.1
- Manno and Sakrison (1974)
high
weight
Contrast sensitivity
low
1
10
100
Spatial frequency (cpd)
47
Conspicuity Map (C_Map)
48
Verification of Model
49
Task Differences
Free-view
50
Head-Mounted Eye-Tracker
Optics module, includes IR source and eye camera
Head-tracking receiver
LASER
Scene camera
External mirror - IR reflecting, visible passing
51
Portable Eye-Tracker
52
Portable Eye-Tracker
53
The Benefits of Eye-Tracking
Newells temporal hierarchy of brain organization
Cognition Working Memory Visual
Routines Neural Operations
10 seconds 1 second 300 msec 80 msec
54
Verification of Model
55
Comparison of Models
56
Task Differences
57
Possible M.S. Projects
  • Comparison of saliency map generation techniques
  • Feature-based
  • Graph-based
  • Information theory-based
  • Object detection from salient keypoints
  • SIFT features
  • Multi-resolution images

58
Possible M.S. Projects
  • Multi-modal tumor classification
  • PET, MRI, CT
  • Solving visual CAPTCHAs
  • Text-based
  • Image-based
  • http//gs264.sp.cs.cmu.edu/cgi-bin/esp-pix

59
Possible M.S. Projects
  • Using reasoning or logic for learning about the
    world from visual observations
  • Bayes nets
  • Reinforcement learning
  • Inductive logic programming (ILP)
About PowerShow.com