Project IST - ARTISTE

1
Work Package 4Image Analysis Algorithms

• Kirk Martinez, Paul Lewis,
• Stephen Chan, Mike Westmacott,
• Mohammed Fauzi, Fazly Abas
• Intelligence, Agents and Multimedia Research
  Group
• Department of Electronics and Computer Science
• University of Southampton
• UK

2
Overview

• The Artiste API
• CCV
• Multiscale CCV
• Histogram
• Multiscale histogram
• Work in progress
• Summary

3
Artiste Image API
Buffer Interface
Artiste Image API
Artiste System
4
Artiste Image API (2)
Buffer Interface
Artiste Image API
5
Artiste Image API - Efforts (3)

• Streamlining the Artiste API for efficient image
  algorithm development
• Images for image analysis are stored in the Vips
  format
• Ports to Windows based machines
• Cygwin Vips dll
• Native Windows dll

6
Colour Coherence Vectors (CCV)
Have we got a similar looking image?
7
CCV Feature Generation (2)
Coherent
Incoherent
Black White Red
Black White Red
8
CCV Benchmarks (3)

• Brute force matching
• Less 2 seconds for any image query on a database
  of over 1100 images

9
Multiscale - CCV (MCCV)
Where does this image fragment come from?
10
MCCV - Aims (2)

• The aims of the M-CCV algorithm
• Find parent images for which sub-images belong
• Find corresponding database images for image
  queries even when images are different
• Find similar images
• Provide accurate location information on where
  the sub-images are found on a parent image

11
MCCV Feature Generation (3)
12
MCCV Rapid Comparison (4)

• 265,000 CCV comparisons per second
• Compressed Vector Comparison

Query FV
Compare
Results
FV0
FV1
FV2
FV3
FVi
Database of feature vectors
13
MCCV Sub-image Location (5)
Image Query
14
MCCV Sub-image Location (6)
Image Query
15
MCCV Benchmarks (7)

• General algorithm
• 17 hours for an image query (130 x 100) image on
  a 512 x 512 sized target image
• Modified algorithm
• 1 minute for an image query (688 x 488 ) on a
  6328 x 4712 sized target image
• Current algorithm
• 45 seconds for an image query (688 x 488) on a
  database of over 1100 images. Size vary from
  440,000 to 30,000,000 pixels

16
Colour Histogram
Have we a got similar looking image?
17
Colour Histogram Feature Generation (2)
Black White Red
18
Colour Histogram Benchmarks (3)

• Brute force matching
• Less 2 seconds for any image query on a database
  of over 1100 images

19
Multiscale Colour Histogram

• Detail finding and similar images
• Similar to the multiscalar approach in MCCV
• Uses Colour histograms instead of CCV
• Faster comparison

20
Work in Progress

• Border Finder and Classification
• MNS
• Finding Faxes
• Plank Detection

21
Border Finder and Classification
Find all paintings of this shape
22
Border Finder and Classification (2)

• Identify and classifying the border of images
• Two stages
• Find border
• Classify border

23
Border Finder and Classification Border
Identification (3)

• Identifies borders by converging sensor points
  from the edge of a painting to the centre

24
Border Finder and Classification Border
Classification (4)

• Border classification by a neural network trained
  to recognise shapes

25
MNS

• What images are like this one?
• Rapid colour matching technique
• Towards painting style classification

26
(No Transcript)
27
MNS Comparison
For all database images
Query Image
More database
Order and rank results
No
images?
Yes
Generate pairwise
Produce stable
Store
Retrieve database
similarity matrix
marriage match
similarity
1
. 0.94
image signature
between query and
between sets, sum
result
database feature sets
pairwise distances
MNS
2
. 0.82
database
3
. 0.53
query
Feature similarity matrix
4
. 0.48

Distance
Sum of the
Summed
between the
distance
threshold
query and the
between
for all
current
matching
unmatched
database
colour pairs
colour pairs
image
(penalty)
Database Images
Query Image
28
(No Transcript)
29
Finding Faxes

• I have a faxed image, can you find me the
  original?
• Locate a colour image from a black and white
  image query
• Technique is based on wavelets
• May provide texture segmentation for other
  algorithms
• Prototype implementation show promising results

30
(No Transcript)
31
(No Transcript)
32
Plank Detection

• Can you find the planks in reverse images?
• Investigating the use of the Hough Transform to
  locate edges which belong to planks

33
Straight Line Detection Using Hough Transform
original image
Accumulator space
extracted edges
Extracted lines
Lines overlaid on image
34
Accumulator Space Cluster Smoothing Algorithm
Accumulator Space Cluster Smoothing Algorithm
Accumulator Space
threshold
Accumulator space
cluster smoothing
After smoothing
Back-mapping Process
Image Space
Lines after smoothing
Lines before smoothing
35
Summary

• Completed algorithms
• CCV, MCCV, Colour Histogram, Multiscale Colour
  Histogram
• Algorithms awaiting integration
• Border finder
• Algorithms in progress
• MNS, Faxes, Plank Detection
• Paper accepted at ICHIM
• Journal paper planned
• Paris 6 (Craddling, Face Location, Query by
  Sketch)

36
Image Algorithm Development Timetable

• http//www.ecs.soton.ac.uk/scyc/iad.htm

37
CCV Retrieval Performance (4)
38
MCCV Retrieval Performance (8)
39
Colour Histogram Retrieval Performance (4)
40
Multiscale Colour Histogram Retrieval
Performance (2)