Title: Semidefinite ranking on graphs Shankar Vembu, Thomas Gaertner, Stefan Wrobel
1Semidefinite ranking on graphsShankar Vembu,
Thomas Gaertner, Stefan Wrobel
2Outline
- Ranking on graphs
- Motivation
- QP relaxation
- Semidefinite ranking
- Experiments and results
3Ranking on graphs - Problem setting
- Input
- Undirected graph
- Directed graph
- Output a permutation
- Few backward edges (directed graph)
- Smooth ordering (undirected graph)
4Ranking on graphs - Optimisation
where ¾ is the step function and º is a
(regularisation) parameter
5Motivation Vertex ordering algorithms
- Ranking of graphs problem is related to several
vertex ordering problems - Minimum bandwidth
- Minimum linear arrangement
- Minimum length ordering
- NP-hard but approximate solutions exist
6Motivation Vertex ordering algorithms
- Idea Generalise one vertex ordering algorithm to
solve the ranking on graphs problem - Issue Vertex ordering problems are unsupervised
- How to incorporate preference constraints?
- Our contribution Modification of the minimum
length ordering algorithm (Blum et al., 00) to
handle preference constraints
7SDP formulations in machine learning
8Graph-based clustering A brief detour
Optimal ratio cut
Cockroach graph
9Spectral relaxation
RatioCut
10Spectral relaxation
Cluster 1
Cluster 2
RatioCut
11Spectral relaxation
1
2
3
4
5
6
7
8
9
10
12
11
20
19
18
17
16
15
14
13
Length of the ordering - 184
12SDP relaxation
RatioCut
13SDP relaxation
Cluster 1
Cluster 2
RatioCut
14SDP relaxation
5
4
3
1
2
11
13
15
19
17
10
9
8
6
7
12
14
16
20
18
Length of the ordering - 173
15QP relaxation (Agarwal, 06)
- Replace the step function with a convex loss
function - Introduce slack variables
- Relax (to real numbers)
16Semidefinite ranking on graphs
- Minimum length ordering problem has an SDP-based
solution with approximation factor O(log2V) - Searches for an embedding of the graph in the
Euclidean space of dimension V - Projects the embedding onto a randomly chosen
vector - Good news The geometry of the embedding could
easily be exploited to incorporate preference
constraints
17Incorporating preference constraints
18The optimisation problem
- Empirical error term
- Regularisation term
- Summands in the empirical error term are not
convex - First summand becomes convex if we change
variables from vectors to inner products between
the vectors
19The optimisation problem
Regulariser
Empirical error term
20Ranking on graphs algorithm
- Solve the semidefinite optimisation problem
- Factorise the Gram matrix using incomplete
Cholesky method - Project the embedding onto a randomly chosen
vector and output the ordering
21Experiments
- Benchmark metric regression data sets
- Converted into ordinal regression data sets by
discretising the target value into equal-lengh
bins - Standardisation (zero mean and unit variance)
- Similarity graphs using Gaussian kernel
- Preference constraints encoded in complete
bipartite graphs between training instances of
successive categories - Inverse 5-fold cross validation
- Kendall tau as the evaluation metric
- DSDP for solving SDP-based ranking
- L-BFGS-B for solving spectral ranking
22Results
23 Future work
- Investigate the use of spreading metrics
- Alternative formulations of the problem
- Low-rank formulations
- Other vertex ordering algorithms (eg minimum
storage-time product) - Large-scale implementations
- Spectral bundle method
24(No Transcript)
25Graph-based clustering A brief detour
- Spectral relaxation (for the RatioCut)
- Solution Fiedler vector (eigenvector
corresponding to the second smallest eigenvalue
of the unnormalised Laplacian L of (V,E))
26Graph-based clustering A brief detour
27SDP relaxation for ranking on graphs
28Incorporating preference constraints
29Motivation - SDP formulations in machine learning
- All the vertex ordering problems mentioned
earlier have SDP-based solutions - Recent studies have shown that SDP relaxations
have superior performance when compared to
spectral relaxations - Clustering algorithms
- Classification algorithms
- Semidefinite relaxation for the ranking on graphs
problem
30SDP formulations in machine learning