INAOE at ImageCLEF2007 Towards Annotation based Image Retrieval - PowerPoint PPT Presentation

1 / 17
About This Presentation
Title:

INAOE at ImageCLEF2007 Towards Annotation based Image Retrieval

Description:

H. Jair Escalante, Carlos Hern ndez, Aurelio L pez, Heidi Mar n, ... Grass, Tree, Rock, Building, People, Mountain, Jet, Sky, Church, Elephant. Set of labels ... – PowerPoint PPT presentation

Number of Views:44
Avg rating:3.0/5.0
Slides: 18
Provided by: MANUELMON2
Category:

less

Transcript and Presenter's Notes

Title: INAOE at ImageCLEF2007 Towards Annotation based Image Retrieval


1
INAOE at ImageCLEF2007Towards Annotation based
Image Retrieval
  • H. Jair Escalante, Carlos Hernández, Aurelio
    López, Heidi Marín, Manuel Montes, Eduardo
    Morales, Enrique Sucar, Luis Villaseñor
  • Language Technologies Laboratory
  • National Institute of Astrophysics, Optics and
    Electronics
  • Tonantzintla, Mexico
  • mmontesg_at_inaoep.mx
  • http//ccc.inaoep.mx/mmontesg

2
Overview of the talk
  • Our first participation at ImageCLEF the goal
    was to build the basic infrastructure
  • Some textual and mixed strategies for image
    retrieval
  • However we could do something more
  • A Web based query expansion method, and
  • An annotation based image retrieval approach

3
Textual and mixed strategies
  • VSM IR System for textual retrieval (baseline)
  • Late fusion of independent retrievers (LF)
  • Intermedia feedback (IMFB)

Topic statement
TBIR
Fusion
Query
RelevantImages
CBIR
Example images
4
Some new things
  • Web-based query expansion
  • Original statement top-k snippets (NQE)
  • Original statement top-l more repeated words
    from the top-k snippets (WQE)
  • Annotation based expansion (ABE)
  • Use automatic image annotation methods for
    obtaining text from images, then
  • Expand documents and/or queries with automatic
    annotations, finally
  • Apply some strategy for textual image retrieval

5
Basis of our idea
  • Region-level annotations are generally
    complementary to manual (image-level) annotations

sky
palm
palm, sky, sand, grass, sea, clouds
clouds
Flamingo Beach Original name in Portuguese
Praia do Flamengo Flamingo Beach is considered
as one of the most beautiful beaches of Brazil
sea
sand
sand
grass
Flamingo Beach Original name in Portuguese
Praia do Flamengo Flamingo Beach is considered
as one of the most beautiful beaches of Brazil
Flamingo Beach Original name in Portuguese
Praia do Flamengo Flamingo Beach is considered
as one of the most beautiful beaches of Brazil
6
Automatic image annotation
  • Assign labels (words) to regions within segmented
    images

Automatic image Annotation method
. . .
Sky
Elephant
Grass 0.6 Sky 0.2 Tree 0.1 Ground 0.1
Annotation improvement
Rock 0.5 Church 0.2 Elephant 0.2 Entrance 0.1
Grass 0.5 Tree 0.3 Ground 0.1 Jet 0.1
Grass
7
Improving the automatic annotation
Grass 0.6 Tree 0.2 rock 0.1 building 0.1
People 0.4 Tree 0.3 Mountain 0.2 Jet 0.1
Tree 0.5 Grass 0.3 Sky 0.1 Jet 0.1
Grass, Tree, Rock, Building, People, Mountain,
Jet, Sky, Church, Elephant
Church 0.3 Grass 0.3 Sky 0.2 Elephant 0.2
8
Set of labels
9
Some problems with the labels
  • 2000 training annotated-regions (2)
  • 98000 regions to annotate (98)
  • Imbalanced training set
  • Limited vocabulary

10
Annotation based query expansion
11
Annotation based document expansion
The surroundings of the Valle Francés Torres del
Paine National Park, Chile March 2002 furniture
grasspeople clouds
The volcano Tungurahua Baños, EcuadorMarch
2002 sand clouds sky mountain
12
Experimental results
Top ranked runs for each configuration
considered.
13
Visual-English run
  • No textual query was used, but at the end the
    recovery was done based on textual data.
  • It combines intermedia feedback and our
    annotation based expansion technique.

14
Textual vs. mixed strategies
15
Initial conclusions
  • Intermedia feedback is an effective way for
    mixing visual and textual information
  • Methods based on web-query expansion showed
    better performance
  • Anotation based expansion is a promising way for
    expanding text using images visual content
  • Annotations can be useful for image retrieval,
    though several issues should be addressed

16
Our current work
  • Work on the improvement of automatic image
    annotation methods
  • Investigate different (better) ways for measuring
    the semantic cohesion between labels and manual
    annotations
  • Use such semantic cohesion estimates for
    improving image retrieval from annotated
    collections

17
Thanks for your attention
Language Technologies Laboratory National
Institute of Astrophysics, Optics and
Electronics Tonantzintla, México Manuel Montes y
Gómez mmontesg_at_inaoep.mx http//ccc.inaoep.mx/mmo
ntesg
Write a Comment
User Comments (0)
About PowerShow.com