Pairwise Constraint Propagation by Semidefinite Programming for SemiSupervised Classification

1
Pairwise Constraint Propagation by Semidefinite
Programming for Semi-Supervised Classification

• Zhenguo Li
• (Joint work with Jianzhuang Liu and Xiaoou Tang)
• Department of Information Engineering
• The Chinese University of Hong Kong

2
Outline

• Semi-Supervised Classification
• Our Work
• Experimental Results
• Conclusions and Future Work

3
Traditional Semi-Supervised Classification

• Learning from labeled and unlabeled data.
• Assumption
• Nearby objects tend to be in the same class
  (cluster assumption).
• Idea
• The known class labels are propagated smoothly to
  unlabeled data (label propagation).

4
Challenges

• The distributions of real-world data are often
  more complex than expected where
• a class may consist of multiple separate groups.
• different classes may be close or overlapped.
• Pairwise constraints are natural in these
  circumstances, which specify whether two objects
  are in the same class or not (must-link and
  cannot-link).
• Techniques for label propagation are not readily
  extended to handle pairwise constraints.

5
Our Work

• We consider the general problem of classifying
  from pairwise constraints and unlabeled data.
• It is more general than traditional
  semi-supervised classification.
• In contrast to label propagation, we attempt to
  explore an approach for pairwise constraint
  propagation.

6
A Toy Classification Example
7
The Global Viewpoint

• The must-link constraint asks to merge the outer
  and inner circles into one class
• The cannot-link constraint asks to keep the
  middle and outer circles into different classes.

8
Our Assumptions

• Cluster Assumption
• Nearby objects should be in the same class.
•  Pairwise Constraint Assumption
• Objects similar to two must-link objects
  respectively should be in the same class
• Objects similar to two cannot-link objects
  respectively should be in different classes.
• Our goal is to implement both the two assumptions
  in a unified framework.

9
Our Idea

• Learn a nonlinear mapping to reshape the data
  such that
• Nearby objects are mapped nearby
• Two must-link objects are mapped close and two
  cannot-link objects are mapped far apart
• Objects similar to two must-link objects
  respectively are mapped close, and objects
  similar to two cannot-link objects respectively
  are mapped far apart. 
• In doing so, the pairwise constraints will be
  propagated to the entire data set.

10
The General Framework
11
Interpretation

• Constraint Satisfaction The inequalities require
  two must-link objects to be mapped close and two
  cannot-link objects to be mapped far apart.
•  Constraint Propagation By enforcing the
  smoothness on the mapping, two objects similar to
  two must-link objects respectively are mapped
  close and two objects similar to two cannot-link
  objects respectively are mapped far apart.
•  After the mapping, hopefully each class becomes
  compact and different classes become far apart.  

12
The Unit Hypersphere Model

• All the objects are mapped onto the unit
  hypersphere. 
• Two must-link objects are mapped to the same
  point.
• Two cannot-link objects to be orthogonal.
• Smoothness measure

13
Learning a Kernel Matrix

• Let
• The matrix can be thought as a
  kernel over the data set, where is just the
  feature map induced by .
• (Kernel Trick) We can implicitly obtain the
  feature map by explicitly pursuing the
  corresponding kernel matrix. 

14
Learning a Kernel Matrix

• The constraints become
•  The smoothness measure becomes
•    

15
The SDP Problem
16
Kernel K-means

• Finally, we apply the kernel K-means to the
  learned kernel matrix to obtain k classes of the
  objects. 

17
Experimental Results Toy Data

• Distance matrices before and after the mapping 

18
Experimental Results UCI Data
19
Experimental Results Image Data
20
Conclusions

• We have proposed a framework PCP for learning
  from pairwise constraints and unlabeled data
• It can effectively propagate pairwise
  constraints
• It is formulated as a SDP problem.
• Future work includes
• accelerating PCP
• handling noisy constraints effectively
• applying PCP to practical applications.

21
Thank You!