# Combinatorial Algorithms - PowerPoint PPT Presentation

PPT – Combinatorial Algorithms PowerPoint presentation | free to download - id: 6a001f-NzkxZ The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Combinatorial Algorithms

Description:

### Introduction. Algorithms in unweighted bipartite graph (Yehong & Gordon) Maximum matching. A simple algorithm. Hopcroft-Karp algorithm. Stable marriage problem (Wang wei) – PowerPoint PPT presentation

Number of Views:8
Avg rating:3.0/5.0
Slides: 99
Provided by: compNusE84
Category:
Tags:
Transcript and Presenter's Notes

Title: Combinatorial Algorithms

1
Combinatorial Algorithms
• (Algorithms in Bipartite Graphs)

2
Outline
• Introduction
• Algorithms in unweighted bipartite graph (Yehong
Gordon)
• Maximum matching
• A simple algorithm
• Hopcroft-Karp algorithm
• Stable marriage problem (Wang wei)
• GaleShapley algorithm
• Algorithms in weighted bipartite graph (Wang
Sheng Jinyang)
• Assignment problem
• Hungarian method Kuhn-Munkres algorithm
• QA

3
Introduction
• Definition
• A graph G (V, E) is bipartite if there exists
partition V X ? Y with X n Y Ø and E ? X Y.
• Bipartite Graph types
• Unweighted
• Weighted
• For every edge e ? E , there is a
• weight w(e) .

4
Introduction
• Example
• There are a set of boys and a set of girls.
• Each boy only likes girls and each girl only
likes boys.
• A common friend wants to match each boy with a
girl such that the boy and girl are both happy
but they both will only be happy if the boy likes
the girl and the girl likes the boy.
• Is it possible for every situation?

We can use a bipartite graph to model this problem
5
Introduction
• Problem
• Testing bipartiteness
• Matching
• Maximum matching problem
• Perfect matching problem
• Stable marriage problem
• Maximum weight matching problem

6
Maximum matching
Yehong
7
Maximum matching
• Definition
• Matching
• A Matching is a subset M ? E such that ?v ? V at
most one edge in M is incident upon v

• Perfect Matching
• A matching which matches all vertices of the graph

8
Maximum matching
free
matched
A Matching
A Maximum Matching
Not a Matching
(not perfect)
• Definition
• We say that a vertex is matched if it is incident
to some edge in M.
• Otherwise, the vertex is free

9
Maximum matching
• Alternating paths
• ( Y1, X2, Y2, X4 )
• Augmenting Path
• (Y1, X2, Y2, X4, Y4, X5)
• Definition
• Alternating Paths
• A path is alternating if its edges alternate
between M and E - M.
• Augmenting Paths
• An alternating path is augmenting if both
endpoints are free
• Alternating Tree
• A tree rooted at some free vertex v in which
every path is an alternating path.

10
Maximum matching
• Property of Augmenting Paths
• Replacing the M edges by the E - M ones
increments size of the matching

(Path Y1, X2, Y2, X4, Y4, X5)
Berge's Theorem A matching M is maximum iff it
has no augmenting path (Proof Lec01 Page 3)
11
Maximum matching
• A simple algorithm

12
Maximum matching
• A simple algorithm

13
Maximum matching
• A simple algorithm

14
Maximum matching
• A simple algorithm

15
Maximum matching
• A simple algorithm
• Commonly search algorithm (BFS, DFS) O(E)
• At most V times
• Complexity O(VE)

16
Hopcroft-Karp Algorithm
• An algorithm to find the maximum matching given a
bipartite graph Gordon

17
Introduction
• The Hopcroft-Karp algorithm was published in 1973
• It is a matching algorithm that finds a maximum
matching in bipartite graphs
• The main idea is to augment along a set of
vertex-disjoint shortest augment paths
simulatenously
• The complexity is O(vVE)
• In this section, some Theorems and Lemmas from
graph theory will be stated without showing the
proof.

18
Definition
• We let the set A ? B denote the symmetric
difference of the set
• A ? B (A ? B) (A n B)
• A maximal set of vertex-disjoint minimum length
augmenting path is defined as follows
• It is a set of augmenting path
• No two path share a same vertex
• If the minimum length augmenting path is of
length k, then all paths in S are of length k
• If p is an augmenting path not in S, then p
shares a vertex with some path p in S

19
Algorithm
The algorithm of Hopcroft and Kraft is as
follows Given a graph G (X ? Y),E) 1) Let
M , 2) Find S P1 , P2 , Pk 3) While S
? M M ? S Find S 4)
Output M
20
Demonstration of algorithm at some stage
Let the dark edges represent the edges in a
matching M
21
Demonstration of algorithm at some stage
Pink edges represent an augmenting path
Deleting them
22
Demonstration of algorithm at some stage
Another augmenting path
No more paths
23
Demonstration of algorithm at some stage
Pink edges represent the paths in maximal set S
M ? S Note the before and after
24
Algorithm
• Question How do we know that this algorithm
produces the result that we want ?
• Theorem 1 (Berge) A matching M is maximum if
and only if there is no augmenting path with
respect to M
• This theorem guarantees the correctness of the
algorithm
• We will now prove that the complexity of the
algorithm is O(vVE)

25
Lemma 2 A maximal set S of vertex-disjoint
minimum length augmenting paths can be found in
O(E) time
Proof Let G (U ? V,E) be the graph that we
are working on and M be a matching
• First , we construct a tree-like/directed
acyclic graph graph given G
level 0

26
Continuation of proof of lemma 2
• Starting at level 2k (even) , the vertices at
level 2k1 are obtained by following free edges
from edges at level 2k
• Starting at level 2k1 (odd) , the level at 2k2
are obtained by following matched edges from
vertices at level 2k1
• Note that the even levels contain vertices from U
and odd levels from V

U
Recall the earlier example There are 3 levels
here
V
27
Continuation of proof of lemma 2
• We continue building the tree until all vertices
have been visited or until a free vertex is
encountered (say t)
• Note that in the latter case, the free vertices
are encountered at V
• Complexity of this portion of building the tree
is linear to the size of the edges ( similar to
BFS)

28
Continuation of proof of lemma 2
Dashed line represent edges while the normal
lines represent edges in the matching M
Example
Free-vertex
0
1
2
3
29
Continuation of proof of lemma 2
• Now we find a maximal set S of vertex disjoint
paths in this tree that we constructed
• We assign a counter to all vertices after level 0
• This counter represents the number of edges
entering the vertex previous level (think of it
like an indegree)
• Starting at a free vertex v at level t, we trace
a path until we reach a free vertex u at level 0

30
Continuation of proof of lemma 2
• This path is an augmenting path and we add it
into S
• After which , we add the vertices in this path
into a deletion queue
• As long as the deletion queue is non empty, we
delete the vertex in the queue and from the
constructed tree
• This includes all the edges incident onto it

Recall the earlier example
31
Continuation of proof of lemma 2
• Whenever an edge is deleted , the counter
associated with its right endpoint are all
decremented
• If the counter becomes 0, put the vertex into the
deletion queue (there can be no augmenting path
from this vertex)
• After emptying the deletion queue, if there are
still free vertex at level t , it means that an
augmenting path must still exist

32
Continuation of proof of lemma 2
• We continue until there are no more free vertex
at level t
• This entire process takes linear time , since it
is proportional to the number of edges deleted
• Therefore this part takes O(E)
• Total time complexity for both parts is O(E)

33
Continuation of proof of lemma 2
Example
Consider the path v6 u6 v5 u1
Deletion Queue
V6 U6 V5 U1
Counter of v1 decreases by 1
34
Continuation of proof of lemma 2
Example
0
0
0
Now consider the path v3 u3 v1 u2
Deletion Queue
V3 U3 V1 U2
V2 U4 V4
35
Theorems and Lemmas cited without proof
• Lemma 3 Let M be a maximum matching, and let
M be any matching in G. If the length of the
shortest augmenting path with respect to M is k,
then M - M (V/k)
• Lemma 4 Let k be the length of the shortest
augmenting path with respect to M and let S be a
maximal set of shortest disjoint augmenting paths
with respect to M, then the length of the
shortest augmenting path with respect to M?S is
larger than k

36
Theorem 5 The Hopcroft-Karp algorithm finds a
maximum matching in a bipartite graph in
O(vVE) time
Proof
• Now we run the algorithm for vV and let M be
matching after running those rounds
• Lemma 4 implies that we have that in each phrase
of the algorithm, the length of the shortest
augmenting path increases by at least 1
• Therefore the size of the shortest augmenting
path must be at least vV

37
Continuation of proof of Theorem 5
• Now from Lemma 3, we have that M - M
(V/vV) vV
• In each phrase , we increase the size of the
matching by at least 1, so therefore , at most
vV more phrases needed
• Therefore at most 2 vV phrases are needed for
this entire algorithm.
• Therefore with lemma 2, the time complexity of
the algorithm is O(vV E)

38
Stable Marriage Problem
• Wang Wei

39
Terminology
• Problem definition
• Given n men and n women, each person has a
preference list for all members of the opposite
sex Find a one-to-one match M.
• If m(a man) and w (a woman) are matched in M,
then m is the partner of w, and vice verse.
• Blocking pair in a match M (m, w), m prefers w
than his partner in M, and w prefers m than her
partner.
• Stable match no blocking pair exist.

40
Stability-checking algorithm
• For each man, try to find a woman, with whom they
form a blocking pair if no such woman exist,
then the match is stable.
• Complexity O(n2)
• Examplehttp//mathsite.math.berkeley.edu/smp/smp.
html

41
Basic Gale-Shapley algorithm
• For man, propose to every women on his preference
list until get engaged
• For woman, wait for proposal, accept if free or
prefer the proposer than current partner/fiance
otherwise reject the proposal
• Complexity O(n2)

42
Theorem 1.
• For any given instance of the stable marriage
problem, the Gale-Shapley algorithm terminates,
and, on termination, the engaged pairs constitute
a stable matching.
• Termination
• Stability if m prefers w than his partner , then
w must have rejected m, i.e., w prefers her
partner to m. (m,w) cannot be a block pair?no
block pair exists.

If GS not terminate, then at least one man is free
To reject a man, the woman must be engaged
He must be rejected by all women
Once a woman is engaged, she will never be free
All women are engaged
All men are engaged
43
• Theorem 2 All possible executions of the
Gale-Shapley algorithm(with the men as the
proposers) yield the same stable matching, in
which, man has the best partner he can have in
any stable matching.
• Theorem 3 In the man-optimal stable matching,
each woman has the worst partner that she can
have in any stable matching.

44
The Hungarian Method
• Jinyang

45
Assignment Problem
• Suppose we have n resources to which we want to
assign to n tasks on a one-to-one basis. Suppose
we also know the cost(gain) of assigning a given
resource to a given task. We wish to ?nd an
optimal assignmentone which minimizes(maximizes)
total cost(gain).
• Min-Cost or Max-Weight Perfect Matching in
Bipartite Graph.

46
Example
• Three students collaborate to finish a project.
Their efficiency is different.

coding test writing
John 4 hours 3 hours 3 hours
Terry 6 hours 4 hours 5 hours
Eric 7 hours 7 hours 5 hours
47
Matrix Representation
• We will formula this problem in a matrix
representation. It is easier to illustrate its
key idea and how it works.
• We will explain how to implement it into
algorithm and show its complexity later.
• We will use minimum cost form of the problem. For
maximum problem, we just reverse the num.

48
Matrix Representation
49
Difficulty
50
Theorem 1
• If a number is added to or subtracted from all of
the entries of any one row or column of a cost
matrix, then on optimal assignment for the
resulting cost matrix is also an optimal
assignment for the original cost matrix.
• You have to choose one entry in each row or
column any way. So this operation add or reduce
the same number for all assignment.

51
Assignment Problem
•

52
Theorem 2
• When there exist a assignment has a zero cost in
a non-negative matrix. This assignment is an
optimal assignment.

53
Hungarian Method
• The key idea of Hungarian Method is to transform
the original matrix to a non-negative matrix
which have a zero assignment by add or subtract
operation in each row and column.
• There will be some slight difference in different
implementation.

54
Hungarian Method
• Step 1 Subtract the smallest entry in each row
from all the entries of its row.
• Step 2 Subtract the smallest entry in each
column from all the entries of its column.
• This two step is not necessary. But it can reduce
the number of iterations later. The only
requirement is that it comes to a non-negative
matrix.

55
Hungarian Method
• Step 3 Draw lines through appropriate rows and
columns so that all the zero entries of the cost
matrix are covered and the minimum number of such
lines is used.
• Step 4 If the minimum number of covering lines
is n, an optimal assignment of zeros is possible
and we are ?nished. Else continue step 5.
• Step 5 Determine the smallest entry not covered
by any line. Subtract this entry from each
uncovered row, and then add it to each covered

56
Example 1
•

57
Example 2
•

58
Example 2
•

59
Example 2
•

60
How to Draw Lines
• The lines is a minimum dominating set of all zero
point.
• Transform the solution of maximum matching to
minimum dominating set.

61
How to Draw Lines

0
0
0
0
62
How to Draw Lines
• Find a maximum assignment(maximum match).

0
0
0
0
63
How to Draw Lines
• Find a maximum assignment(maximum match).
• Mark all rows having no assignments .

0
0
0
0
64
How to Draw Lines
• Find a maximum assignment(maximum match).
• Mark all rows having no assignments . Then mark
all columns having zeros in marked row(s).

0
0
0
0
65
How to Draw Lines
• Find a maximum assignment(maximum match).
• Mark all rows having no assignments . Then mark
all columns having zeros in marked row(s). Then
mark all rows having assignments in marked
columns .Repeat this until a closed loop is
obtained.

0
0
0
0
66
How to Draw Lines
• Find a maximum assignment(maximum match).
• Mark all rows having no assignments . Then mark
all columns having zeros in marked row(s). Then
mark all rows having assignments in marked
columns .Repeat this until a closed loop is
obtained.
• Then draw lines through all marked columns and
unmarked rows.

0
0
0
0
67
Why Always Stop?
•

68
Kuhn-Munkres Algorithm the Polynomial Hungarian
Method
• Wang Sheng

69
Evolvement of Hungarian Method
• 1955, Harold Kuhn
• Hungarian method was published and was largely
based on the earlier works of two Hungarian
mathematicians
• 1957, James Munkres
• Munkres observed it is polynomial in O(n4) and
since then the algorithm was also known as
Kuhn-Munkres algorithm
• 1960, Edmonds and Karp
• The KM algorithm was modified to achieve an O(n3)
running time

70
Introduction to KM Algorithm
• Basic Hungarian method
• Consider assignment problem in terms of matrix
• Idea add/subtract X from all entries of a
row/column
• Goal choose 0s from nonnegative matrix
• Easy to understand
• Kuhn-Munkres algorithm
• Consider assignment problem in terms of bipartite
graph
• Easy to analysis and implement
• Our Goal
• Introduce KM and show both of them are equivalent

71
Restate Assignment Problem
• Matrix vs. Bipartite graph
• For each entry Ci,j in matrix, there is an edge
in bipartite graph from Xi to Yj with weight
equal to Ci,j
• In order to be consistent with theorems
introduced in the algorithm we consider
max-weight matching

72
Definitions
• Feasible labeling L
• A vertex labeling is a function L V ? R
• A feasible labeling is one such that L(x)L(y)
w(x, y), ?x ? X, y ? Y
• Equality Graphs
• Equality Graph is G (V,EL) where EL (x, y)
L(x)L(y) w(x, y)

73
Kuhn-Munkres Theorem
• If L is feasible and M is a perfect matching in
EL then M is a max-weight matching
• each node is covered exactly once and L(x)L(y)
w(x, y) therefore the upper-bound of the weight
is the sum of labels
• Power of the theorem
• Transform problem from weighted matching to
un-weighted perfect matching

74
Inspiration from KM Theorem
• Key idea
• find a good feasible labeling that remains enough
edges in equality graph to ensure perfect
matching can be done
• Algorithm proposal
matching M in EL
• While M is not perfect, repeat Find an
augmenting path in EL to increase the size of
M or if no path exists, improve L to L such that
EL ? EL

75
Finding an Initial Feasible Labeling
• Simplest assignment
• Maximize L(x) while Minimize L(y) ?y ? Y, L(y)
0 ?x ? X, L(x) maxw(x,y), y ? Y
• It is obvious that ?x ? X, y ? Y, w(x, y)
L(x)L(y)

76
Improving Labeling
• Neighbor of u ? V and S ? V
• NL(u) v (u,v) ? EL
• NL(S) ?u ? S NL(u)
• Lemma
• Let S ? X and T NL(S) ? Y.
• Set aL min L(x) L(y) w(x,y) , x ? S, y ?
T
• Update the labels (1) if v ? S L(v) L(v) -
aL (2) if v ? T L(v) L(v) aL (3)
otherwise L(v) L(v)
• Then L is a feasible labeling

77
Equivalence of Graph and Matrix
• aL min L(x) L(y) w(x,y) , x ? S, y ? T
• Consider a matrix C with w(x,y) as its elements
• For row x ? X / column y ? Y, add L(x)/ L(y) to
each element
• Problem is equivalent to solve the min-cost
assignment in C
• Edge with L(x) L(y) w(x,y) is the 0 element
in matrix C
• (1) if v ? S, L(v) L(v) - aL (2) if v ? T,
L(v) L(v) aL
• S set of uncovered rows T set of covered
columns
• aL the smallest entry not covered by any line
• Subtract this entry from each uncovered row, and
then add it to each covered column

78
Effectiveness of Label Update
• Edges in EL (1) if v ? S L(v) L(v) - aL (2)
if v ? T L(v) L(v) aL (3) otherwise L(v)
L(v)
• If (x,y) ? EL for x ? S, y ? T then (x,y) ? EL
• If (x,y) ? EL for x ? S, y ? T then (x,y) ? EL
• There is some edge (x,y) ? EL for x ? S, y ? T
• With good choice of S, we can guarantee there are
more edges in new Equality Graph

79
Kuhn-Munkres Algorithm
1. Generate initial labeling L and matching M in EL
2. If M is perfect, terminate. Otherwise pick free
vertex u ? X. Set S u, T .
3. If NL(S) T, update labels(forcing NL(S) ?
T) aL min L(x) L(y) w(x,y) , x ? S, y ?
T (1) if v ? S L(v) L(v) - aL (2) if v ?
T L(v) L(v) aL (3) otherwise L(v) L(v)
4. If NL(S) ? T, pick y ? NL(S) T. If y is free,
augmenting u-y and go to 2. If y is matched to z,
S S ? z, T T ? y. Go to 3.

80
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

81
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

82
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

83
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

84
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

85
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

86
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

87
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

88
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

89
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

90
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

91
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

92
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

93
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

94
Example
• Generate initial labeling and matching
• Pick a free vertex, set Su T otherwise
stop
• If NL(S) T, update labels (force NL(S) ? T)
• If NL(S) ? T, pick y to be NL(S) T
• If y is free, augment u y, go to step 2 If
y is matched to z, S S ? z, T T ? y. go
to step 3

max-weight is 16
95
Complexity
• In each phase of algorithm, M increases by 1,
so there are at most V phases.
• ?y ? T keep track of slacky minL(x)L(y)-w(x,y)
• In each phase
• Initializing all slacks. O(V)
• When a vertex moves into S, all slacks need
update. O(V) Only V vertices can be moved into
S. O(V2)
• When updating labels, aL min(slacky).
O(V) After getting aL, must update slacky
slacky -aL. O(V) aL can be calculated V times
per phase. O(V2)
• Total time per phase is O(V2)
• Total running time is O(V3)

96
Q A
97
References
• Maximum matching / Hopcroft-Karp Algorithm
• http//en.wikipedia.org/wiki/Matching_(graph_theor
y)
• http//www.cs.dartmouth.edu/ac/Teach/CS105-Winter
05/Notes/kavathekar-scribe.pdf
• http//en.wikipedia.org/wiki/HopcroftE28093Karp
_algorithm
• http//www2.informatik.huberlin.de/alkox/lehre/lvs
s12/ga/notes/HK.pdf
• http//www.dis.uniroma1.it/leon/tcs/lecture2.pdf

98
References
• Stable Matching Problem