Feature Combination and Relevance Feedback for 3D Model Retrieval

1
Feature Combination and Relevance Feedback for 3D
Model Retrieval
The Eleventh International Multi-Media Modelling
Conference

• Indriyati Atmosukarto1,2, Wee Kheng Leow1,
  Zhiyong Huang1
• 1 School of Computing, National University of
  Singapore
• 2 Department of Computer Science, University of
  Washington

2
Introduction

• Retrieval of 3D models have attracted much
  research interest
• Funkhouser et al, ACM TOG 03
• Kazhdan et al, EG Symp on Geometry Processing 03
• Tangelder and Veltkamp, SMI 03
• Yu et al, CVPR 03
• Osada et al, SMI 02
• Elad et al, EGMM 01
• Suzuki et al, IEEE Conf SMC 00
• Ankerst et al, Int Symp on Spatial DB 99
• Kriegel et al, Int Symp on Large Spatial DB 97
• Besl and McKay, IEEE Trans PAMI 92

3
Our Work

• A query is defined in terms of a set of at least
  one object instead of some shape features
• The query processing is based on pairwise
  rankings of the objects, which are monotonically
  related to the pairwise distances
• Query processing becomes more efficient
• Use less space without dimensionality reduction
• Avoid explicit computation in high dimensional
  feature spaces
• The contributions of known relevant and
  irrelevant objects are not combined using
  weighted sum, instead, our method ensures that
  known relevant objects always rank at the top and
  known irrelevant objects always rank at the
  bottom
• The weights of various feature types are computed
  by approximating the probability that the feature
  type is effective in retrieving relevant objects

4
Object Matching Criteria

• Our criterion of 3D shape similarity is expressed
  succinctly by the well-known principle of rigid
  object registration
• The error of registration

5
3D Object Representation Schemes

• 3D grids
• 2D Spherical map
• Histogram

6
3D Shape Features

• Distance of the points from the objects centroid
• Local elongation
• Bumpiness
• Total curvature
• Gaussian curvature
• Random distance
• Random angle
• Random area

7
Representation Schemes vs. Shape Features

• All the features can be captured in 3D grids, 2D
  maps, and histograms to represent objects in
  different forms
• For 3 representations and 8 features, there are
  24 different combinations
• Our test results show only the combinations using
  13 features give good results (next slide)

8
The Good Combinations

• Histograms of random distance (RDH), random angle
  (RAH), random area (RRH), and Gaussian curvature
  (KH). Each histogram has 200 bins
• 2D spherical maps of distance (DM), elongation
  (EM), bumpiness (BM), total curvature (TM),
  Gaussian curvature (KM), and random distance
  (RDM). Each map contains 64x64entries
• 3D grids of elongation (EG), bumpiness (BG), and
  Gaussian curvature (KG). Each 3D grid contains
  25x25x25entries

9
Memory Consideration

• Storing the pairwise distances of up to 5500
  objects actually requires less memory space than
  storing the 13 feature representation
• The memory space required to store these 13
  feature representations of an object is
  4x206x64x643x25x25 72.3x103 unit
• N objects Nx72.3x10103 units
• Suppose we pre-compute the pairwise distances
  between two objects measured according to each of
  the feature representations, then the total
  memory space required is 13N2.
• The break-even point is then 5500

10
Object Ranking

• We compute the integer rank rk(OiOj) of object
  Oi wrt Oj in increasing order of the distance
  dk(Oi,Oj) according to the representation k
• The test results show the weighted sum of the
  distances do not yield good retrieval performance

11
A Query

• A query Q is defined in terms of a set of
  relevant object RRj (cannot be f) and a set of
  irrelevant object IIj (can be f)
• The query Q is specified by the user
• The query processing task is to determine the
  similarity between an object Oi in the database
  and the query Q

12
Object Similarity

• For representation k
• If If
• If I?f

Xk is the exclusive set. It is derived from I
(details in the paper)
13
Overall Object Similarity

• si?k wksik / ?k wksik
• wk is the weight of feature representation k
• wk reflects the probability that representation k
  is effective in retrieval
• wk is estimated as the ratio of the number of
  known relevant objects over the number of objects
  within the hypersphere in the feature space
  spanned by the know relevant objects (details in
  the paper)

14
Test Database

• The database is created by merging three existing
  sets of objects
• The first dataset contained 52 objects from 34
  categories. Among these 52 objects, 6 of them
  were manually articulated with the help of 3D
  StudioMax to produce a total of 110 articulated
  objects
• The second set, the Utrecht Database, contained
  512 aircrafts in six categories delta jets,
  conventional airplanes, multifuselages, biplanes,
  helicopters, and other aircrafts
• The third set is a subset of the Princeton
  Database. It contained 1236 objects in 52
  categories
• In total, our test database contained 1910
  objects.

15
The System GUI
16
Test Procedure

• First, a user selected one relevant object to
  form the query set
• Next, the system retrieved and displayed the top
  48 objects, and the user selected the relevant
  objects
• Then, this retrieval and feedback process was
  repeated until no new relevant object was
  retrieved
• For retrieval by single features, no irrelevant
  object were selected. These tests were used to
  obtain the baseline results
• For the feature combination method, two types of
  tests were performed with and without irrelevant
  objects
• In the tests where irrelevant objects were used,
  all objects displayed on the GUI that were not
  marked as relevant by the user were regarded as
  irrelevant. The type of selected relevant objects
  reflects the users query context

17
Test Results

• Table 1 of the paper
• For example, retrieving humans in a fixed posture
  (fHu) requires rigid object matching criterion,
  whereas retrieving humans in any posture (Hu)
  requires articulated object matching criterion
• Other rigid objects include head (He), guitar
  (Gu), computer monitor (Mo), rifle (Ri), and
  pistol (Pi). Other articulated objects include
  hand (Ha), ant (An), eagle (Ea), and shark (Sh)

18
Video
19
Conclusion

• A novel method of combining various feature
  representations and relevance feedback processing
• It performs query processing based on known
  relevant and irrelevant objects in the query
• It computes the similarity of an object with the
  query using pre-computed rankings of the objects
• It uses less space
• Query processing is very efficient
• Test results show that the feature combination
  method significantly improves the retrieval
  performance of individual feature types

20
Acknowledgment

• MMM 2005 anonymous reviewers
• This research is supported by ARF
  R-252-000-137-112
• Lu Haiyun and Huang Wenfan for the system
  implementation