Content Based Image Retrieval - PowerPoint PPT Presentation

1 / 35
About This Presentation
Title:

Content Based Image Retrieval

Description:

show me images of snow covered mountains. sketch shape required. similarity based ... Draw a horse returns dogs, cows etc. Draw side view of cars doesn't ... – PowerPoint PPT presentation

Number of Views:2196
Avg rating:3.0/5.0
Slides: 36
Provided by: cem96
Category:

less

Transcript and Presenter's Notes

Title: Content Based Image Retrieval


1
Content Based Image Retrieval
  • Images generated by
  • satellites, military reconnaissance/surveillance
    flights, fingerprinting, mug shots,
  • scientific experiments, biomedical imaging
  • Applications are
  • Art Galleries and Museum Management
  • Interior Design, Remote Sensing, Weather
    forecasting, Picture Archiving, Fabric and
    Fashion Design, Law Enforcement and Criminal
    Investigation, Scientific/Medical Database
    Management

2
Previous Approaches
  • Set of attributes extracted manually and managed
    by standard DBMS
  • Feature extraction / object recognition system
  • Automated approaches to object recognition are
    computationally expensive, difficult and tend to
    be domain specific

3
  • 2 categories of features
  • Primitive/Low level
  • Logical
  • Images can be retrieved by
  • colour texture
  • sketch shape
  • motion text
  • objective attributes subjective attributes
  • browsing

4
CBIR
  • CBIR must also address broader info retrieval
    context
  • medical images / diagnosis / treatment
  • Query specification
  • needs natural language
  • show me images of snow covered mountains
  • sketch shape required
  • similarity based

5
Query by Image Content(QBIC)
  • QBIC allows queries on large image and video
    databases based on
  • example images
  • user sketches and drawings
  • colour and texture patterns
  • demo at http//wwwqbic.almaden.ibm.com/

6
QBIC
  • Database population
  • images and videos are processed to extract
    features - colours, textures, shapes, camera and
    object motion
  • Database query
  • User comprises graphical query
  • Features generated from query and input to
    matching engine that finds image or video with
    similar features

7
  • Illustration of a histogram for a grey scale
    image
  • This is what information the computer sees
  • You can tell from the histogram that the image is
    dark but nothing about the content.

8
Histogram matching
  • Store histogram for each image
  • Compare images using some statistical method
  • Return sorted matches best fit first
  • Problem - These would have the same histogram

9
Red Histogram
10
Green Histogram
11
Blue Histogram
12
Colour match
  • Average colour
  • adds up red, green, blue components of each pixel
  • Colour position
  • 6x8 or 9x12 grid overlays picture. For each block
    average Munsell colour and 5 most frequently
    occurring colours

QBIC interprets the virtual canvas as a grid of
coloured areas, then matches this grid to other
images stored in the database.
13
Shape matching
Aggregate shape measurement
Easier to automatically compute shape if
background is plain This was produced by
PaintShopPro
14
Textures
  • Textures can be rough or smooth, vertical or
    horizontal etc

15
QBIC
  • Queries can be
  • Find images with approximately 30 red, 15 blue
  • Find images with red round object

Drawn query
Best match
Matches in decreasing order of correlation
16
QBIC
  • Queries can be
  • Find images similar to this one
  • Find sunsets with 30 red, 30 yellow
  • Find images with 30 red and blue textured object
  • Find images like this sketch

User sketch
Best match
17
Problems with shapes
  • What would you input to retrieve both of these?
  • Other problems
  • Draw a horse returns dogs, cows etc
  • Draw side view of cars doesnt retrieve front
    views
  • Objects may be partially obscured

18
Colour similarity
All histograms should be similar to picture 1
19
Colour similarity and keyword
All histograms should be similar to picture
1 but keyword man was included. The picture of
the lion has the keyword MANE! The
following picture has keyword MANUFACTURING!
20
Colour Layout
Colour position should be similar to picture 1
21
Colour percentage
34 yellow 30 red 36 dont care
22
Colour percentage and keyword
34 yellow 30 red 36 dont care keyword SUNSET
23
Keyword only
keyword SUNSET
24
Custom Paint Query
Top two thirds of picture blue Lower
third yellow
25
Texture search
Texture search like first picture
26
Texture search
Texture search like first picture
27
Performance and usability
  • Effectiveness
  • measure of relevance of retrieved images
  • Efficiency
  • system responsiveness and interactivity of system
  • Usability
  • measuring and evaluating human performance and
    preference

28
Sound retrieval by content
  • Current applications, Soundfisher for example
    search using the following attributes
  • Amplitude
  • Pitch
  • Duration
  • Sample rate
  • These attributes are not very user friendly
  • Soundfisher finds similar soundbites but does not
    work well for voice or musical data

29
Sound retrieval interface
  • How can you input your search data
  • Use audio file (WAV, MP3, MID)
  • Use keyboard (assumes some musical ability)
  • Whistle / hum/ sing (assumes some musical
    ability)
  • A major problem with searching a sound file is
    that the same melody played on different
    instruments will give a different sound wave

30
Sound retrieval by content
  • Same music different instruments
  • How can you perform a search for this melody?

31
Video Searching
  • How to locate individual sequences from
  • a database of video clips
  • a long piece of video ( a whole movie)
  • Can be done by
  • using a textual database of the clips
  • fast-forwarding through movie

32
Using Storyboards
  • One approach is to use a storyboard representing
    the movie.
  • Storyboard consists of a series of video stills
    representing the movie but much smaller in size.
  • Storyboards can be browsed manually or using a
    query by example

33
Reverse Engineering the Video
  • Storyboards need to be generated from the
    finished video sequence
  • Each shot requires
  • start point, end point and most representative
    picture
  • first two can be found by comparing the amount
    and constancy of change between individual frames

34
Most Representative Frame
  • The average frame can be generated by using an
    average colour for each pixel
  • The best representative frame was the one that
    differed most from the average
  • Storyboards could be used over networks when it
    is time consuming to download a complete video

35
Storyboard example
Original storyboard from The Wrong Trousers
Storyboard generated from the video The Wrong
Trousers
Write a Comment
User Comments (0)
About PowerShow.com