Separatosr in the Real World Practice

1
Separatosr in the Real World Practice

• Slides combined from
• www.cs.berkeley.edu/demmel/cs267_Spr07
• www.cs.cmu.edu/guyb/realworld/slidesF03/src/separ
  ators1.ppt

2
Edge Separators

• An edge separator a set of edges E ½ E which
  partitions V into V1 and V2
• Criteria
• V1, V2 balanced
• E is small

V1
V2
E
3
Vertex Separators

• An vertex separator a set of vertices V ½ V
  which partitions V into V1 and V2
• Criteria
• V1, V2 balanced
• V is small

2
0
V2
1
V1
3
6
4
5
7
8
4
Other names

• Sometimes referred to as
• graph partitioning (probably more common than
  graph separators)
• graph bisectors
• graph bifurcators
• balanced graph cuts

5
Application

• Telephone network design
• Original application, algorithm due to Kernighan
• Load Balancing while Minimizing Communication
• Sparse Matrix times Vector Multiplication
• Solving PDEs
• N 1,,n, (j,k) in E if A(j,k) nonzero,
• WN(j) nonzeros in row j, WE(j,k) 1
• VLSI Layout
• N units on chip, E wires, WE(j,k) wire
  length
• Sparse Gaussian Elimination
• Used to reorder rows and columns to increase
  parallelism, and to decrease fill-in
• Data mining and clustering
• Physical Mapping of DNA

6
Recursive Separation
7
What graphs have small separators

• Planar graphs O(n1/2) vertex separators
• 2d meshes, constant genus, excluded minors
• Almost planar graphs
• the internet, power networks, road networks
• Circuits
• need to be laid out without too many crossings
• Social network graphs
• phone-call graphs, link structure of the web,
  citation graphs, friends graphs
• 3d-grids and meshes O(n1/2)

8
What graphs dont have small separatos

• Expanders
• Can someone give a quick proof that they dont
  have small separators
• Hypercubes
• O(n) edge separators O(n/(log n)1/2) vertex
  separators
• Butterfly networks
• O(n/log n) separators ?
• It is exactly the fact that they dont have small
  separators that make them useful.

9
Applications of Separators

• Circuit Layout (dates back to the 60s)
• VLSI layout
• Solving linear systems (nested dissection)
• n3/2 time for planar graphs
• Partitioning for parallel algorithms
• Approximations to certain NP hard problems
• TSP, maximum-independent-set
• Clustering and machine learning
• Machine vision

10
More Applications of Separators

• Out of core algorithms
• Register allocation
• Shortest Paths
• Graph compression
• Graph embeddings

11
Available Software

• METIS U. Minnessota
• PARTY University of Paderborn
• CHACO Sandia national labs
• JOSTLE U. Geenwich
• SCOTCH U. Bordeaux
• GNU Popinet
• Benchmarks
• Graph Partitioning Archive

12
Different Balance Criteria

• Bisectors 50/50
• Constant fraction cuts e.g. 1/3, 2/3
• Trading off cut size for balance
• min cut criteria
• min quotient separator
• All versions are NP-hard

13
Other Variants of Separators

• k-Partitioning
• Might be done with recursive partitioning, but
  direct solution can give better answers.
• Weighted
• Weights on edges (cut size), vertices (balance)
• Hypergraphs
• Each edge can have more than 2 end points
• common in VLSI circuits
• Multiconstraint
• Trying to balance different values at the same
  time.

14
Edge vs. Vertex separators

• If a class of graphs satisfies an f(n)
  edge-separator theorem then it satisfies an f(n)
  vertex-separator.
• The other way is not true (unless degree is
  bounded)

15
Separator Trees
16
Asymptotics

• If S is a class of graphs closed under the
  subgraph relation, then
• Definition S satisfies a f(n) vertex-separator
  theorem if there are constants a lt 1 and b gt 0 so
  that for every G ? S there exists a cut set C ?
  V, with
• C lt b f(G) cut size
• A lt a G, B lt a G balance
• Similar definition for edge separators.

17
Separator Trees

• Theorem For S satisfying an (a,b) n1-e edge
  separator theorem, we can generate a perfectly
  balanced separator tree with separator size C
  lt k b f(G).
• Proof by picture C b n1-e(1 a a2 )
  b n1-e(1/1-a)

18
Algorithms

• All are either heuristics or approximations
• Kernighan-Lin (heuristic)
• Planar graph separators (finds O(n1/2)
  separators)
• Geometric separators (finds O(n(d-1)/d)
  separators)
• Spectral (finds O(n(d-1)/d) separators)
• Flow techniques (give log(n) approximations)
• Multilevel recursive bisection (heuristic,
  currently most practical)

19
Partitioning via Breadth First Search

• BFS identifies 3 kinds of edges
• Tree Edges - part of T
• Horizontal Edges - connect nodes at same level
• Interlevel Edges - connect nodes at adjacent
  levels
• No edges connect nodes in levels
• differing by more than 1 (why?)
• BFS partioning heuristic
• N N1 U N2, where
• N1 nodes at level lt L,
• N2 nodes at level gt L
• Choose L so N1 close to N2

BFS partition of a 2D Mesh using center as root
N1 levels 0, 1, 2, 3 N2 levels 4, 5, 6
20
Kernighan-Lin Heuristic

• Local heuristic for edge-separators based on
  hill climbing. Will most likely end in a
  local-minima.
• Two versions
• Original takes n2 times per step
• Fiduccia-Mattheyses takes n times per step

21
High-level description for both

• Start with an initial cut that partitions the
  vertices into two equal size sets V1 and V2
• Want to swap two equal sized sets X ?A and Y ? B
  to reduce the cut size.

C
A
B
Y
X
Note that finding the optimal subsets X and Y
solves the optimal separator problem, so it is NP
hard. We want some heuristic that might help.
22
Some Terminology

• C(A,B) the weighted cut between A and B
• I(v) the number of edges incident on v that
  stay within the partition
• E(v) the number of edges incident on v that go
  to the other partition
• D(v) E(v) - I(v)
• D(u,v) D(u) D(v) - 2 w(u,v)
• the gain for swapping u and v

A
v
C
B
u
23
Kernighan-Lin improvement step

• KL(G,A0,B0)
• ? (u ? A0, v ? B0 )
• put (u,v) in a PQ based on D(u,v)
• for k 1 to V/2
• (u,v) max(PQ)
• (Ak,Bk) (Ak-1,Bk-1) swap (u,v)
• delete u and v entries from PQ
• update D on neighbors (and PQ)
• select Ak,Bk with best Ck
• Note that can take backward steps (D(u,v) can be
  negative).

A
v
C
B
u
24
Fiduccia-Mattheyses improvement step

• FM(G,A0,B0)
• ?u ? A0 put u in PQA based on D(u)
• ?v ? B0 put v in PQB based on D(v)
• for k 1 to V/2
• u max(PQA)
• put u on B side and update D
• v max(PQb)
• put v on A side and update D
• select Ak,Bk with best Ck

A
v
C
B
u
25
A Bad Example for KL or FM

• Consider following graph with initial cut given
  in red.

2
2
2
1
1
1
KL (or FM) will start on one side of grid and
flip pairs over moving across the grid until the
whole thing is flipped. The cut will never be
better than the original bad cut.
26
Performance in Practice

• In general the algorithms do very well at
  smoothing a cut that is approximately correct.
• Used by most separator packages either
• to smooth final results
• to smooth partial results during the algorithm
• K/L tested on very small graphs (Nlt360) and
  got convergence after 2-4 sweeps
• For random graphs (of theoretical interest) the
  probability of convergence in one step appears to
  drop like 2-N/30

27
Introduction to Multilevel Partitioning

• If we want to partition G(N,E), but it is too big
  to do efficiently, what can we do?
• 1) Replace G(N,E) by a coarse approximation
  Gc(Nc,Ec), and partition Gc instead
• 2) Use partition of Gc to get a rough
  partitioning of G, and then iteratively improve
  it
• What if Gc still too big?
• Apply same idea recursively

28
Multilevel Partitioning - High Level Algorithm
(N,N- ) Multilevel_Partition( N, E )
recursive partitioning routine
returns N and N- where N N U N-
if N is small (1) Partition G
(N,E) directly to get N N U N-
Return (N, N- ) else (2)
Coarsen G to get an approximation Gc
(Nc, Ec) (3) (Nc , Nc- )
Multilevel_Partition( Nc, Ec ) (4)
Expand (Nc , Nc- ) to a partition (N , N- ) of
N (5) Improve the partition ( N ,
N- ) Return ( N , N- )
endif
(5)
V - cycle
(2,3)
(4)
How do we Coarsen? Expand? Improve?
(5)
(2,3)
(4)
(5)
(2,3)
(4)
(1)
29
Multilevel Kernighan-Lin

• Coarsen graph and expand partition using maximal
  matchings
• Improve partition using Kernighan-Lin

30
Maximal Matching Example
31
Example of Coarsening
32
Coarsening using a maximal matching
1) Construct a maximal matching Em of G(N,E) for
all edges e(j,k) in Em 2) collapse
matched nodes into a single one Put node
n(e) in Nc W(n(e)) W(j) W(k) gray
statements update node/edge weights for all nodes
n in N not incident on an edge in Em 3) add
unmatched nodes Put n in Nc do not
change W(n) Now each node r in N is inside a
unique node n(r) in Nc 4) Connect two nodes in
Nc if nodes inside them are connected in E for
all edges e(j,k) in Em for each other
edge e(j,r) or (k,r) in E Put edge
ee (n(e),n(r)) in Ec W(ee)
W(e) If there are multiple edges
connecting two nodes in Nc, collapse them,
adding edge weights
33
Expanding a partition of Gc to a partition of G
34
Multilevel Spectral Bisection

• Coarsen graph and expand partition using maximal
  independent sets
• Improve partition using Rayleigh Quotient
  Iteration

35
Example of Coarsening
- encloses domain Dk node of Nc
36
Coarsening using Maximal Independent Sets
Build domains D(k) around each node k in Ni
to get nodes in Nc Add an edge to Ec whenever
it would connect two such domains Ec empty
set for all nodes k in Ni D(k) ( k,
empty set ) first set contains nodes
in D(k), second set contains edges in D(k) unmark
all edges in E repeat choose an unmarked
edge e (k,j) from E if exactly one of k
and j (say k) is in some D(m) mark e
add j and e to D(m) else if k and j
are in two different D(m)s (say D(mi) and
D(mj)) mark e add edge (mi,
mj) to Ec else if both k and j are in the
same D(m) mark e add e to
D(m) else leave e unmarked
endif until no unmarked edges
37
Expanding a partition of Gc to a partition of G

• Need to convert an eigenvector vc of L(Gc) to an
  approximate eigenvector v of L(G)
• Use interpolation

For each node j in N if j is also a node in
Nc, then v(j) vc(j) use same
eigenvector component else v(j)
average of vc(k) for all neighbors k of j in
Nc end if endif
38
Example 1D mesh of 9 nodes
39
Improve eigenvector Rayleigh Quotient Iteration
j 0 pick starting vector v(0) from
expanding vc repeat jj1 r(j)
vT(j-1) L(G) v(j-1) r(j)
Rayleigh Quotient of v(j-1)
good approximate eigenvalue v(j) (L(G) -
r(j)I)-1 v(j-1) expensive to do
exactly, so solve approximately using an
iteration called SYMMLQ, which uses
matrix-vector multiply (no surprise) v(j)
v(j) / v(j) normalize v(j) until
v(j) converges Convergence is very fast cubic
40
Example of convergence for 1D mesh
41
Available Implementations

• Multilevel Kernighan/Lin
• METIS (www.cs.umn.edu/metis)
• ParMETIS - parallel version
• Multilevel Spectral Bisection
• S. Barnard and H. Simon, A fast multilevel
  implementation of recursive spectral bisection
  , Proc. 6th SIAM Conf. On Parallel Processing,
  1993
• Chaco (www.cs.sandia.gov/CRF/papers_chaco.html)
• Hybrids possible
• Ex Using Kernighan/Lin to improve a partition
  from spectral bisection

42
Comparison of methods

• Compare only methods that use edges, not nodal
  coordinates
• CS267 webpage and KK95a (see below) have other
  comparisons
• Metrics
• Speed of partitioning
• Number of edge cuts
• Other application dependent metrics
• Summary
• No one method best
• Multi-level Kernighan/Lin fastest by far,
  comparable to Spectral in the number of edge cuts
• Spectral give much better cuts for some
  applications
• Ex image segmentation
• See Normalized Cuts and Image Segmentation by
  J. Malik, J. Shi

43
Number of edges cut for a 64-way partition
For Multilevel Kernighan/Lin, as implemented in
METIS (see KK95a)
Expected cuts 6427 2111 1190
11320 3326 4620 1746 8736
2252 4674 7579
3D mesh 31805 7208 3357 67647
13215 20481 5595 47887 7856
20796 39623
of Nodes 144649 15606 4960
448695 38744 74752 10672 267241
17758 76480 201142
of Edges 1074393 45878
9462 3314611 993481 261120 209093 334931
54196 152002 1479989
Edges cut 88806 2965
675 194436 55753
11388 58784 1388 17894
4365 117997
Graph 144 4ELT ADD32 AUTO BBMAT FINAN512 LHR10 M
AP1 MEMPLUS SHYY161 TORSO
Description 3D FE Mesh 2D FE Mesh 32 bit
adder 3D FE Mesh 2D Stiffness M. Lin. Prog. Chem.
Eng. Highway Net. Memory circuit Navier-Stokes 3D
FE Mesh
Expected cuts for 64-way partition of 2D mesh
of n nodes n1/2 2(n/2)1/2 4(n/4)1/2
32(n/32)1/2 17 n1/2 Expected cuts
for 64-way partition of 3D mesh of n nodes
n2/3 2(n/2)2/3 4(n/4)2/3
32(n/32)2/3 11.5 n2/3
44
Thank you!