Saliency Detection and Matching for PhotoIdentification of Humpback Whales

1
Saliency Detection and Matching for
Photo-Identification of Humpback Whales

• Elena Ranguelova, Eric Pauwels
• Center for Mathematics and Computer Science
  (CWI),
• Amsterdam, The Netherlands

www.cwi.nl/ely/projects.htm
2
Overview

• Introduction
• System overview
• Segmentation
• Retrieval using grid features
• Retrieval using salient patterns
• Salient pattern detection
• Pattern representation
• Pattern matching
• Combined retrieval
• Results
• Conclusions

3
Introduction

• Goal- computer-assisted photo-identification of
  marine mammals
• Challenges
• Large databases
• Image quality, viewing angle,
  occlusion

4
System overview
Salient pattern detector
5
Segmentation

• Semi-automatic tail extraction
• Initial marker
• Marker-controlled watershed segmentation
• Fine-tuning with additional markers
• Clipping tail to minimize occlusion
• Patch segmentation
• Gray-level thresholding
• Local fine-tuning

6
Phluke_Phinder

• http//homepages.cwi.nl/ely/software.htm

7
Coordinate grid

• Grid construction
• Anatomical landmarks
• Affine invariance
• regions

8
(No Transcript)
9
Salient pattern detector
Salient pattern detector

• Saliency operator based on morphology operators

10
(No Transcript)
11
Pattern representation
12
Pattern matching

• Goal finding correspondence between two sets of
  salient patterns
• Query ,DB image
• Correspondence ,vector
• Probability of correspondence
• Probability of no match
• Global correspondence with local consistency
• Potentially
  and
• Dissimilarity
• Weights
• Consistency- and

13
Matching algorithm

• Find all potential correspondences for
  all patterns in the query with all
  patterns in the database
• For all correspondences compute and initial
  probabilities
• Iterate by updating the probabilities
• using the quality of correspondence
• for all pairs of consistent correspondences.
• Normalize

14
Combined retrieval

• Salient pattern similarity score
• Combined similarity score

15
Results

• Test database
• 340 humpback fluke images of 150 individuals
  (each with 2 to 4 different photos)
• Image quality (1- excellent, 2- good/moderate,
  3-poor)
• photographic (85 high/good, 15 poor)
• recognition (7 poor)
• both (2.5 poor)
• Segmentation- for images of high photographic
  quality in one iteration, few iterations of
  fine-tuning for the rest

16
Results salient pattern detection
17
Results pattern matching
18
Results- retrieval I
Table Percentage of images whose first true
match is ranked amongst the top k
19
Results- retrieval II

• Robustness to image quality

20
Phluke_Matcher
21
Conclusions

• System for photo-identification of humpback
  whales
• Semi-automatic segmentation of tail and patches
• Binary salient detector
• Matching of salient patterns
• General-purpose tools