Content-Based Image Retrieval - Approaches and Trends of the New Age

1
Content-Based Image Retrieval - Approaches and
Trends of the New Age

• Ritendra Datta, Jia Li, and James Z. Wang
• The Pennsylvania State University
• MIR2005

2
INTRODUCTION

• ???image??????text???
• Text is mans creation, images are a mere replica
  of what man has seen
• Interpretation of what we see is hard to
  characterize
• visual similarity ! semantic similarity
• CBIR has grown tremendously after 2000, not just
  in terms of size, but also in the number of new
  directions explored

3
INTRODUCTION
4
INTRODUCTION

• The theoretical foundation behind how we humans
  interpret images is still an open problem
• A brief scanning of about 300 relevant papers
  published in the last five years revealed that
  less than 20 were concerned with applications or
  real-world systems

5
CBIR??????

• Feature Extraction
• Approaches to Retrieval
• Annotation and Concept Detection
• Relevance Feedback and Learning
• Hardware and Interface Support

6
Feature Extraction

• ??? Color Feature
• An Efficient Color Representation for Image
  Retrieval (???histograms?)
• Multiresolution Histograms and Their Use for
  Recognition (??textured image)
• Image retrieval using color histograms generated
  by Gauss mixture vector quantization
  (??GMVQ?color histogram)

7
Feature Extraction

• Color Texture ??
• Wavelet-Based Texture Retrieval Using
  Generalized Gaussian Density and Kullback-Leibler
  Distance
• Shape
• Shape Matching and Object Recognition Using
  Shape Contexts (is fairly compact yet robust to
  a number of geometric transformations)

8
Feature Extraction

• Segmentation
• Normalized Cuts and Image Segmentation
  (????????)
• Blobworld Image Segmentation Using
  Expectation-maximization and Its Application to
  Image Querying (????????)
• Segmentation of Brain MR Images Through a Hidden
  Markov Random Field Model and the
  Expectation-Maximization Algorithm (??medical
  imaging)

9
Feature Extraction

• ?????
• Image retrieval using wavelet-based salient
  points
• ????feature
• Application-specific feature sets (????)
• SIMPLIcitySemantics-Sensitive Integrated
  Matching for Picture Libraries
  (semantics-sensitive feature selection)
• Feature Selection for SVMs (?classifier)

10
Approaches to Retrieval

• Region based image retrieval
• A Scalable Integrated Region-Based Image
  Retrieval System
• region-based querying (BlobWorld)
• Vector quantization (VQ) on image blocks
• Keyblock An Approach for Content-based Image
  Retrieval (generate codebooks for representation
  and retrieval, taking inspiration from data
  compression and text-based strategies)

11
Approaches to Retrieval

• Windowed search
• Object-Based Image Retrieval Using the
  Statistical Structure of Images (more effective
  than methods based on inaccurate segmentation)
• Anchoring-based image retrieval
• A Study of Image Retrieval by Anchoring
  (Anchoring is based on the idea of finding a set
  of representative anchor images and deciding
  semantic proximity between an arbitrary image
  pair in terms of their similarity to these
  anchors)

12
Approaches to Retrieval

• Probabilistic frameworks for image retrieval
• A Probabilistic Architecture for Content-based
  Image Retrieval

13
Annotation and Concept Detection

• Supervised classification
• Image Classification for Content-Based Indexing
  (involving simple concepts such as city,
  landscape, sunset,and forest, have been achieved
  with high accuracy)
• Translation approach
• Object recognition as machine translation
  Learning a lexicon for a fixed image vocabulary
  (???clef 2004??follow???)

14
Annotation and Concept Detection

• ??????
• We humans segment objects better than machines,
  having learned to associate over a long period of
  time, through multiple viewpoints, and literally
  through a streaming video at all times
• The association of words and blobs become truly
  meaningful only when blobs isolate objects well

15
Relevance Feedback and Learning

• Relevance Feedback in Image Retrieval A
  Comprehensive Review
• Problems
• One problem with RF is that after every round of
  user interaction, usually the top results with
  respect to the query have to be recomputed
• Another issue is the users patience in
  supporting multi-round feedbacks

16
REAL-WORLD REQUIREMENTS

• Performance
• Semantic learning
• Volume of Data
• Concurrent Usage
• Heterogeneity
• Multi-modal features
• User-interface
• Operating Speed
• System Evaluation

17
CURRENT RESEARCH TRENDS

• Journals
• IEEE T. Pattern Analysis and Machine Intelligence
  (PAMI)
• IEEE T. Image Processing (TIP)
• IEEE T. Circuits and Systems for Video Technology
  (CSVT)
• IEEE T. Multimedia (TOM)
• J. Machine Learning Research (JMLR)
• International J. Computer Vision (IJCV)

18
CURRENT RESEARCH TRENDS

• Pattern Recognition Letters (PRL)
• ACM Computing Surveys (SURV)
• Conferences
• IEEE Computer Vision and Pattern Recognition
  (CVPR)
• International Conference on Computer Vision
  (ICCV)
• European Conference on Computer Vision (ECCV)
• IEEE International Conference on Image Processing
  (ICIP)

19
CURRENT RESEARCH TRENDS

• ACM Multimedia (MM)
• ACM SIG Information Retrieval (IR)
• ACM Human Factors in Computing Systems (CHI)

20
(No Transcript)
21
(No Transcript)
22
CONCLUSIONS

• We have presented a brief survey on work related
  to the young and exciting fields of content-based
  image retrieval and automated image annotation,
  spanning 120 publications in the current decade
• We have laid out some guidelines for building
  practical, real-world systems