Semidefinite ranking on graphs Shankar Vembu, Thomas Gaertner, Stefan Wrobel

1
Semidefinite ranking on graphsShankar Vembu,
Thomas Gaertner, Stefan Wrobel
2
Outline

• Ranking on graphs
• Motivation
• QP relaxation
• Semidefinite ranking
• Experiments and results

3
Ranking on graphs - Problem setting

• Input
• Undirected graph
• Directed graph
• Output a permutation
• Few backward edges (directed graph)
• Smooth ordering (undirected graph)

4
Ranking on graphs - Optimisation

where ¾ is the step function and º is a
(regularisation) parameter
5
Motivation Vertex ordering algorithms

• Ranking of graphs problem is related to several
  vertex ordering problems
• Minimum bandwidth
• Minimum linear arrangement
• Minimum length ordering
• NP-hard but approximate solutions exist

6
Motivation Vertex ordering algorithms

• Idea Generalise one vertex ordering algorithm to
  solve the ranking on graphs problem
• Issue Vertex ordering problems are unsupervised
• How to incorporate preference constraints?
• Our contribution Modification of the minimum
  length ordering algorithm (Blum et al., 00) to
  handle preference constraints

7
SDP formulations in machine learning
8
Graph-based clustering A brief detour
Optimal ratio cut
Cockroach graph
9
Spectral relaxation
RatioCut
10
Spectral relaxation
Cluster 1
Cluster 2
RatioCut
11
Spectral relaxation
1
2
3
4
5
6
7
8
9
10
12
11
20
19
18
17
16
15
14
13
Length of the ordering - 184
12
SDP relaxation
RatioCut
13
SDP relaxation
Cluster 1
Cluster 2
RatioCut
14
SDP relaxation
5
4
3
1
2
11
13
15
19
17
10
9
8
6
7
12
14
16
20
18
Length of the ordering - 173
15
QP relaxation (Agarwal, 06)

• Replace the step function with a convex loss
  function
• Introduce slack variables
• Relax (to real numbers)

16
Semidefinite ranking on graphs

• Minimum length ordering problem has an SDP-based
  solution with approximation factor O(log2V)
• Searches for an embedding of the graph in the
  Euclidean space of dimension V
• Projects the embedding onto a randomly chosen
  vector
• Good news The geometry of the embedding could
  easily be exploited to incorporate preference
  constraints

17
Incorporating preference constraints
18
The optimisation problem

• Empirical error term
• Regularisation term
• Summands in the empirical error term are not
  convex
• First summand becomes convex if we change
  variables from vectors to inner products between
  the vectors

19
The optimisation problem
Regulariser
Empirical error term
20
Ranking on graphs algorithm

• Solve the semidefinite optimisation problem
• Factorise the Gram matrix using incomplete
  Cholesky method
• Project the embedding onto a randomly chosen
  vector and output the ordering

21
Experiments

• Benchmark metric regression data sets
• Converted into ordinal regression data sets by
  discretising the target value into equal-lengh
  bins
• Standardisation (zero mean and unit variance)
• Similarity graphs using Gaussian kernel
• Preference constraints encoded in complete
  bipartite graphs between training instances of
  successive categories
• Inverse 5-fold cross validation
• Kendall tau as the evaluation metric
• DSDP for solving SDP-based ranking
• L-BFGS-B for solving spectral ranking

22
Results
23
Future work

• Investigate the use of spreading metrics
• Alternative formulations of the problem
• Low-rank formulations
• Other vertex ordering algorithms (eg minimum
  storage-time product)
• Large-scale implementations
• Spectral bundle method

24
(No Transcript)
25
Graph-based clustering A brief detour

• Spectral relaxation (for the RatioCut)
• Solution Fiedler vector (eigenvector
  corresponding to the second smallest eigenvalue
  of the unnormalised Laplacian L of (V,E))

26
Graph-based clustering A brief detour

• Semidefinite relaxation

27
SDP relaxation for ranking on graphs
28
Incorporating preference constraints
29
Motivation - SDP formulations in machine learning

• All the vertex ordering problems mentioned
  earlier have SDP-based solutions
• Recent studies have shown that SDP relaxations
  have superior performance when compared to
  spectral relaxations
• Clustering algorithms
• Classification algorithms
• Semidefinite relaxation for the ranking on graphs
  problem

30
SDP formulations in machine learning