Pilot presentation RICH

1

• Pilot presentation RICH

2
Introduction

• RICH Reading Images for the Cultural Heritage
• Two pilot projects
• Application for semi-automatic dendrochronology
• Content-based image retrieval for historical
  glass
• Today, focus on the latter project

3
Introduction

• Classification of archaeological artefacts
• Now performed manually by experts
• Expert compares artefact with objects from
  reference collection
• Reference collections consist of drawings in
  books
• Thus slow, subjective, and error-prone process

4
Example

• An archaeological artefact

5
Example

• And its corresponding drawing
• ?

6
The task

• Given an artefact photograph
• Find the most similar drawings
• Speeds up the classification process
• Can give archaeological experts new insights

7
The problem

• Drawings contain no color information
• Drawings contain only very abstract texture
  representation
• Texture hard to extract from glass photographs
• Thus only outer shape information is accurate
• Shape matching

8
Shape matching

• Several approaches in literature
• Shape contexts (Belongie, 2000)
• Curvature scale spaces (Mokhtarian, 1996)
• Turning functions (Tanase, 2003)
• Dynamic programming (Petrakis, 2002)
• Moment invariants (Hu, 1962)
• Hausdorff, Procrustes, etc.
• MPEG-7 standard Curvature scale spaces

9
Shape matching

• For today, focus on
• Shape contexts
• Curvature scale space
• Desired properties of approach
• Invariant to scale, translation, and rotation
• Robust to distortions due to
• Broken artefacts
• 3D rotations
• Drawing interpretations

10
Broken artefacts

• ?

11
Or worse
12
3D rotations

• ?

13
Shape contexts

• Sample points from outer contour
• For all points
• Compute angle (relative to baseline) and distance
  to all other points in coarsely discretized
  log-polar space
• All resulting histograms form the shape
  representation

14
Shape contexts

• Matching using startpoint invariant k-NN
  classifier (using ?2-distance)
• Startpoint invariance obtained by circular
  shifting one of the histograms

15
Curvature scale space

• Determine positions of zero-crossings of
  curvature for an evolving shape contour
• Curvature is a function that is 0 for a straight
  line, and 1 / r for a circle with radius r
• Shape evolution convolve coordinates with 1D
  Gaussian kernel with increasing variance

16
Shape evolution

• Evolving shape with curvature zero-crossings

17
Curvature scale space

• CSS image
• Align CSS by aligning global maximum
• Sum distances between main peaks

18
Results

• Low identification performance (as expected) due
  to difficult dataset

19
Results

• We examined various variations, such as
  quantization in shape context space, etc.
• Best performance 33 for hitlist size 10

20
Results

• However
• For good artefacts results are encouraging
• Shape analysis on reference collection allows for
  making shape maps (using MDS)
• This allows for archaeologists to create new
  typologies (since typologies are not fixed)
• Allows for new methods of presenting collections
• Good results expected for flint data

21
Example query
?
22
(No Transcript)
23
Applications

• Matlab application (local)

24
Applications

• Navigation structure for collection presentation
• Precalculated and stored in database
• http//www.referentiecollectie.nl/ri
  chglas

25
Applications

• Web-based CBIR tool
• Servlet running on webserver (using Tomcat)
• User uploads photograph to webserver
• Sends query image to UM calculation server (using
  RMI)
• RMI server executes original Matlab-scripts
  (using JMatLink)
• Results are sent back to servlet
• Servlet generates result pages
• Advantages no local calculations, no porting of
  code

26
Conclusions

• Matching glass artefacts with drawings is a
  difficult problem
• Shape context matching outperforms (MPEG-7
  standard) CSS matching
• Allows for shape analysis of reference collection
• Number of (preliminary) applications delivered
• We expect shape matching to be usefull for flint
  data
• Possible improvements
• Incorporating texture features
• Design of shape matching methods for partial
  shape matching using closed contours

27
Questions

• ?