Context - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Context

Description:

Context – PowerPoint PPT presentation

Number of Views:22
Avg rating:3.0/5.0
Slides: 44
Provided by: learInr
Category:
Tags: aie | context

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Context


1
Context
  • or no context

Rob Fergus (out of context) Derek Hoiem (in
context) Antonio Torralba (in context)
2
The Question Lottery
  • Everyone gets to write one question
  • Drawn randomly from bag
  • Put your name on the paper
  • Copy from your neighbours!
  • One condition
  • Need to have time after the slides

3
Who needs context anyway? We can recognize
objects even out of context
Banksy
4
The face detection age
  • Human Face Detection in Visual Scenes - Rowley,
    Baluja, Kanade (1995)
  • Graded Learning for Object Detection - Fleuret,
    Geman (1999)
  • Robust Real-time Object Detection - Viola, Jones
    (2001)
  • Feature Reduction and Hierarchy of Classifiers
    for Fast Object Detection in Video Images -
    Heisele, Serre, Mukherjee, Poggio (2001)

5
Face detection and the Head in the coffee beans
problem
The success of face detection has induced people
to detect other objects using the same
philosophy objects are detected
independently They have created the head in the
coffee beans problem.
6
Head in the coffee beans problem
Can you find the head in this image?
7
Head in the coffee beans problem
Can you find the head in this image?
8
Relational violations
False alarms occur in image regions in which is
impossible for the target to be present.
9
Just plain incoherence
10
The system does not care about the scene, but we
do
We know there is a keyboard present in this scene
even if we cannot see it clearly.
11
The multiple personalities of a blob
Torralba IJCV 2003
12
The multiple personalities of a blob
Torralba IJCV 2003
13
Palmer 1975
  • Scene preceded object to identify.
  • Better identification when preceded by a
    semantically consistent scene.

Objects seen for 20, 40, 60 or 120 ms.
14
Loftus Mackworth
  • Inconsistent objects fixated earlier and longer.
  • Suggested additional processing of objects out of
    context.
  • Similar results found by Friedman (1979).

15
The context challenge
How far can you go without using an object
detector?
16
The context challenge
What do you think are the hidden objects?
1
2
17
The context challenge
What do you think are the hidden objects?
Answering this question does not require knowing
how the objects look like. It is all about
context.
18
The context challenge
  • Predict
  • Is the object present?
  • What is the most probable locations?
  • What is the most probable object appearance?

Extract scene features Do not use a model of the
object !
  1. Extract scene features
  2. Learn p(object present scene features)

19
Global features can predict expected
locations/scales of objects before running
detectors
Keyboards
Pedestrians
There is a relationship between the aspect of the
objects in a scene, and the aspect of the scene
itself. For instance, the point of view of cars
is correlated with the orientation of the street.
But also, the location of the ground in the scene
is correlated with the location of the objects in
the scene.
Torralba Sinha 2001 Murphy, Torralba, Freeman
2004
20
Levels of context
  • Context in low-level vision
  • Part-based models
  • Objects relations

Fix graph structures can be useful approximations
Long-range connections Weak constraints Multimodal

21
Context representations
The dimensionality grows fast by making the
window big.
22
Detecting difficult objects
Maybe there is a mouse
Office
Start recognizing the scene
Torralba, Murphy, Freeman. NIPS 2004.
23
Detecting difficult objects
Detect first simple objects (reliable detectors)
that provide strong contextual constraints to the
target (screen -gt keyboard -gt mouse)
Torralba, Murphy, Freeman. NIPS 2004.
24
Detecting difficult objects
Detect first simple objects (reliable detectors)
that provide strong contextual constraints to the
target (screen -gt keyboard -gt mouse)
Torralba, Murphy, Freeman. NIPS 2004.
25
Context defines function
From intrinsic features
A car out of context is less of a car
From contextual features
Torralba, Murphy, Freeman. NIPS 2004.
26
An Age of Scene Understanding
Ohta Kanade 1978
  • Guzman (SEE), 1968
  • Hansen Riseman (VISIONS), 1978
  • Barrow Tenenbaum 1978
  • Brooks (ACRONYM), 1979
  • Marr, 1982
  • Ohta Kanade, 1978
  • Yakimovsky Feldman, 1973

27
Objects and Scenes
  • Biedermans violations (1981)

28
Support
Golconde Rene Magritte
29
Interposition
Blank Check Rene Magritte
30
Size
The Listening Room Rene Magritte
31
Position, Probability
Personal Values Rene Magritte
32
Previous work on context
  • Strat Fischler (91)
  • Context defined using hand-written rules about
    relationships between objects

33
Object-Object Relationships
  • Enforce spatial consistency between labels using
    MRF

Carbonetto, de Freitas Barnard (04)
34
Object-Object Relationships
  • Use latent variables to induce long distance
    correlations between labels in a Conditional
    Random Field (CRF)

He, Zemel Carreira-Perpinan (04)
35
Object-Object Relationships
  • Fink Perona (NIPS 03)
  • Use output of boosting from other objects at
    previous iterations as input into boosting for
    this iteration

36
Object-Object Relationships
Torralba Murphy Freeman 2004
Kumar Hebert 2005
37
3d Scene Context
Features that do not encode specific object
information
Confusion Matrix (in using Layout template)
Classification of prototypical scenes (400 /
category)
Degree of Expansion
Degree of Openness
38
3d Scene Context
Torralba Murphy Freeman 2003
39
3d Scene Context
Image
Support
Vertical
Sky
V-Center
V-Left
V-Right
V-Porous
V-Solid
Hoiem, Efros, Hebert ICCV 2005
40
3d Scene Context
Image
World
41
3d Scene Context
Ped
Ped
Car
Hoiem Efros Hebert 2006
42
Hierarchical Sharing and Context
E. Sudderth, A. Torralba, W. T. Freeman, and A.
Wilsky.
  • Scenes share objects
  • Objects share parts
  • Parts share features

See Eriks poster today and tomorrow
43
Issues
  • Datasets
  • LabelMe
  • MIT street scenes
  • MSRC
  • Feedforward vs. semantic context
  • Other types of context
About PowerShow.com