Models of Greedy Algorithms for Graph Problems

1
Models of Greedy Algorithms for Graph Problems

• Sashka Davis, UCSD
• Russell Impagliazzo, UCSD
• SIAM SODA 2004

2
Why greedy algorithms?

• Greedy algorithms are simple, and have efficient
  implementations
• They are used as
• Exact algorithms for many optimization problems
• Approximation schemes with guaranteed
  approximation ratios for hard problems
• As heuristics for hard optimization problems

3
Goal

• To design an abstract model of greedy algorithms
  and answer
• the questions
• Could a known greedy approximation algorithm be
  improved?
• Can we prove lower bounds on approximation ratio
  of greedy algorithms for hard problems?
• Can we formalize the intuition that greedy
  algorithms are weaker than other algorithmic
  paradigms?

4
History

1. BNR02 defined Priority algorithm framework for
   scheduling problems
2. AB02 defined Priority algorithms for facility
   location and set cover
3. BL03 proved bounds on performance of Priority
   algorithms for VC, IS and Coloring

5
General results

• Extended the work of BNR02 and AB02 and
  defined problem-independent model for greedy
  algorithms
• Defined a formal model of Memoryless priority
  algorithms
• Characterization of the power of Fixed,
  Memoryless, and Adaptive algorithms in terms of
  combinatorial games

6
Separations
Fixed Priority Algorithms
Dynamic Programming Algorithms
Adaptive Priority Algorithms
Memoryless Priority Algorithms
7
Results for graph problems

• Shortest paths
• No Fixed priority algorithm can achieve any
  constant approximation ratio, for ShortPath
  problem on graphs, with non-negative weights
• No Adaptive priority algorithm can achieve any
  constant approximation ratio, for ShortPath
  problem on graphs with negative weights, but no
  negaitve weight cycles

8
Results for specific graph problems

• Shortest paths
• No Fixed priority algorithm can achieve any
  constant approximation ratio, for ShortPath
  problem on graphs, with non-negative weights
• No Adaptive priority algorithm can achieve any
  constant approximation ratio, for ShortPath
  problem on graphs with negative weights, but no
  negaitve weight cycles

9
Results for specific graph problems

• Steiner trees
• Proved lower bound of 1.18 on the approximation
  ratio achieved by Adaptive priority algorithms
• Improved adaptive priority algorithm, achieving
  an approximation ratio 1.875 for special metric
  instances where the distance between nodes is
  1,2

10
Results for specific graph problems

• Weighted Vertex Cover
• Proved lower bound 2 for Adaptive priority
  algorithms, matching the standard
  2-approximation scheme
• Independent Set
• Proved lower bound of 3/2 on the performance of
  Adaptive algorithms for degree-3 graphs

11
Remainder of the talk

• What is a greedy algorithm?
• Formal definitions of Fixed and Adaptive priority
  algorithms and show a strong separation between
  Fixed and Adaptive priority algorithms
• Lower bound 2 on the approximation ratio of
  Adaptive priority algorithms on weighted vertex
  cover problem

12
What is a greedy algorithm?

• Given a universe of data items ?
• The instance is a set of data items, subset of ?
• The algorithm defines an ordering function on ?
  and views the data items in the instance in that
  order
• The algorithm makes an irrevocable decision for
  each data item, which depends only on data items
  seen so far, and decisions made so far, not
  future data items
• The solution is decisions made on each data item

13
Kruskal algorithm for MST

• Input (G(V,E), ? E ?R)
• Initialize empty solution T
• L Sorted list of edges in non-decreasing order
  according to their weight
• while (L is not empty)
• e next edge in L
• Add the edge to T, as long as T remains a forest
  and remove e from L
• Output T

14
Fixed priority algorithms

• ? is a set of data items ? is a set of options
• Input instance I?1 ,?2 ,,?n , I? ?
• Output solution S(?i , ?i) i 1,2,,d ?i ?
  ?
• 1. Determine an ordering function p ??R ?8
• 2. Order I according to p(?)
• 3. Repeat
• Let the next data item in the ordering p be ?i
• Make a decision ?i ? ?
• Update the partial solution S
• until (decisions are made for all data items)
• 4. Output S(?i , ?i) i 1,2,,d

15
Kruskal is a Fixed priority algorithm

• ? is a set of edges
• Each edge is represented as (u, v, ?)
• ?accepted, rejected
• Priority function p (u, v, ?) ?

16
Question

• Can all problems with known greedy algorithms be
  solved by a Fixed priority algorithm?
• ShortPath Problem Given a graph G(V,E),
• ? E ?R s, t ?V. Find a directed tree of
  edges,
• rooted at s, such that the combined weight of the
• path from s to t is minimal

17
Answer
ShortPath
Fixed Priority Algorithms

• Theorem No Fixed priority algorithm can achieve
  any constant approximation ratio for the
  ShortPath problem

18
Fixed priority game
?i2
?i9,
?i1
?i3
?i4
?i5
?i6
?i7
?i8
G0
G1
G2
G3
Ø
si2
si4
End Game S_adv (?i2,si2), (?i4,si4)
S_sol (?i2,si2)
S_sol (?i2,si2), (?i4,si4)
19
Fixed priority game for ShortPath problem

• Data items are edges of the graph
• Decision options accept, reject
• A strategy for the Adversary in the game
  establishes a lower bound on approximation ratio
  achieved by any Fixed priority algorithm

20
Adversary selects ?0
21
Solver selects an order on ?0

• If then the Adversary
  presents

u(k)
a
y(1)
t
s
x(1)
z(1)
b
22
Adversarys strategy

• Waits until Solver considers edge y(1)
• Solver will consider y(1) before z(1)

23
Event 1 Solver accepts y(1)
24
Event 2 Solver rejects y(1)
u(k)
a
y(1)
t
s
x(1)
b
The Solver fails to construct a path. The
Adversary outputs a solution u,y.
25
The outcome of the game

• The Solver either fails to output a solution or
  achieves an approximation ratio (k1)/2
• The Adversary can set k arbitrarily large and
  thus can force the Algorithm to claim arbitrarily
  large approximation ratio

26
Conclusion

• No Fixed priority algorithm can achieve any
  constant approximation ratio for the ShortPath
  problem
• Dijkstra algorithm solves the ShortPath problem
  exactly
• Dijkstra algorithm (G(V,E), s ? V)
• T?Ø S?s
• Until (S?V)
• Find e(u,x) e mine?Cut(S, V-S)path(s,
  x)?(e)
• T? T?e S ? S ?x

27
Adaptive priority algorithms

• ? is a set of data items ? is a set of options
• Input instance I?1, ?2,,?d , I??
• Output solution S(?i , ?i) i 1, 2,,d
• 1. Initialization UØ SØIØt1
• 2. Repeat
• Determine an ordering function pt ?-I? R?8
• Pick the highest priority data item ?t?U
  according to pt
• Make an irrevocable decision ?t ? ?
• Update U ? U-?t S?S ? (?t, ?t) I?I ?
  ?t t?t1
• until (decisions are made for all data items)
• 3. Output S(?i , ?i) i 1,2,,d

28

• Dijkstra is an Adaptive priority algorithm
• ? is a set of edges Each edge is represented as
  (u, v, ?)
• ?accepted, rejected
• ordering

29
Weighted Vertex Cover

• Joh74 gave a greedy 2-approximation algorithm
• Can we design an improved approximation algorithm
  in the class of Adaptive priority algorithms?

30
Joh74 greedy 2-approximation for WVC

• Input instance I?1, ?2,,?n
• Output solution S ? I
• 1. Initialization UØ SØIØt1
• 2. Repeat
• pt (?) ?(v)/(adj_list I)
• Order all ?t?U according to their pt value in
  non-decreasing order. Let ?t be first data item
  in the order.
• if (pt (?) ? 8) then
• Make an irrevocable decision ?t in
• Update U ? U-?t S?S ? ?t I?I ? ?t
  t?t1
• until (pt (?) ? 8)
• 3. Output S

31
Priority model for WVC problem

• A data item is a vertex of the graph
  ? (v, ?(v), adj_list )
  ?V ?R
• The input to the algorithm is a set of data items
  I?1, ?2,,?n
• Set of decision options is Saccept, reject
• Ordering function pt (?) ?(v)/(adj_list
  I)
• For simplicity, the solution will be a vertex
  cover S ? I, of vertices accepted by the
  algorithm.

32
Question

• Is there an Adaptive priority algorithm which
• gives a better for the WVC than the known
• 2-approximation? No!
• Theorem No Adaptive priority algorithm can
• achieve an approximation ration better than 2

33
Adaptive priority game
G3
G1
G2
G0
?9
?10
?12
?3
?2
?7
?6
?8
?
?5
?11
?1
?4
S_sol (?7,s7)
S_sol (?7,s7), (?4,s4)
s4
s7
s2
S_sol (?7,s7), (?4,s4),(?2,s2)

• The Game Ends
• S_adv (?7,s7), (?4,s4),(?2,s2)
• Solver is awarded payoff
• f(S_sol)/f(S_adv)

34
The Adversary chooses instances to be graphs Kn,n

• The weight function ?V? 1, n2

35
The game

• Data items
• each node appears in ?o as two separate data
  items with weights 1, n2
• Solver moves
• Choses a data item, and commits to a decision
• Adversary move
• Removes from the next ?t the data item,
  corresponding to the node just committed and..

36
Adversarys strategy is to wait unitl

• Event 1 Solver accepts a node of weight n2
• Event 2 Solver rejects a node of any weight
• Event 3 Solver has committed to all but one
  nodes on either side of the bipartite

37
Event 1 Solver accepts a node ?(v)n2
1
1
1
1
1

• The Adversary chooses part B of the bipartite as
  a cover, and incurs cost n
• The cost of a cover for the Solver is at least
  n2n-1

38
Event 2 Solver rejects a node of any weight
n2
n2

• The Adversary chooses part A of the bipartite as
  a cover.
• The Solver must choose part B of the bipartite as
  a cover.

39
Event 3 Solver commits to n-1 nodes w(v)1, on
either side of Kn,n
1
1
1
1
1
n2
1

• The Adversary chooses part B of the bipartite as
  a cover, and incurs cost n
• The cost of a cover for the Solver is 2n-1

40
Summary

• No Adaptive priority algorithm can achieve an
  approximation ratio better than 2 for the WVC
• The known 2-approximation is optimal and cannot
  be improved

41
Conclusions

1. The class of Adaptive priority algorithms is more
   powerful
2. The known 2-approximation for the WVC is optimal
   in the class of Adaptive priority algorithms and
   cannot be improved

42
Future research directions

• Extend the framework to capture larger class of
  greedy algorithms
• Define a notion of global information
• Redefine the notion of local information
• Build formal models for backtracking and dynamic
  programming algorithms and evaluate their
  performance on hard problems