UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2002

1
UMass Lowell Computer Science 91.503 Analysis
of Algorithms Prof. Karen Daniels Fall, 2002

• Lecture 6
• Tuesday, 10/8/02
• NP-Completeness

2
Overview

• NP-Completeness
• Motivation
• Reduction to Establish Lower Bounds
• Basics
• Literature
• How to Treat NP- Hard or Complete Problems
• Midterm Exam

3
Motivation
4
Why do we care??

• When faced with a problem in practice, need to
  know whether or not one can expect produce a
  fast algorithm (i.e. O(nk)) to solve it
• Intractable problems occur frequently in
  practice
• Surface differences between tractable and
  intractable problems are often hard to see

5
UMass Lowell Algorithms Lab
OS 305
Prof. Daniels
Manufacturing
Data Mining Visualization
Telecommunications
6
Industrial MotivationSupporting Apparel
Manufacturing
7
Dynamic Covering Motivation

• sensor coverage
• clustering
• shape recognition
• channel
• assignment
• production allocation
• facility location
• layout

8
New Covering Research Objective

• Practical solutions to dynamic hard core covering
  problems
• dynamic instance sequence of static instances
• hard even static problem is computationally
  intractable
• core useful in multiple applications
• covering assignment allocation
• Practical solve static instancei quickly if it
  is easy
• combinatorially small or
• clearly infeasible (for decision problem) or
• clearly feasible (for decision problem) or
• a small change from instancei-1

Geometric or Combinatorial
Feasibility or Optimization
9
State-of-the-Art Dynamic Algorithms Data
Structures

• Polynomial-time problems
• Data structures supporting fast updates
• Self-adjusting (e.g. splay trees) Sleator, CMU
• Kinetic Data Structures Guibas, Stanford
• Hard problems
• Polynomial-time approximation schemes for
  optimization problems Greedy can be sensitive
  to small changes Gao, et al.
• Meta-heuristics no theoretical guarantee
• Practical algorithms for small and/or easy
  instances
• Containment decision problem Daniels,
  Milenkovic
• successful building block for layout problems
• leverages strong combinatorial optimization
  techniques
• Distributed heuristics
• Primarily for dynamic network routing or channel
  assignment
• Wu et al. connected dominating sets for network
  routing
• no theoretical guarantee some empirical support
• Randomized, distributed
• Gao, et al. discrete mobile centers
  (clustering)
• theoretical framework assumes knowledge of
  change trajectories

static
10
State-of-the-Art Covering Problems
covering
non-geometric covering
geometric covering
VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic
minimization, facility location
2D translational covering
P finite point sets
P shapes
Q identical

• Thin coverings of the plane with congruent convex
  shapes
• Translational covering of a convex set by a
  sequence of convex shapes
• Translational covering of nonconvex set with
  nonidentical covering shapes

Q nonconvex
Q convex
1D interval covered by annuli
BOX-COVER
partition

• NP-hard/complete polygon problems
• polynomial-time results for restricted orthogonal
  polygon covering and horizontally convex polygons
• approximation algorithms for boundary, corner
  covers of orthogonal polygons

Polynomial-time algorithms for triangulation and
some tilings
11
New Covering Research Example 1 Translational
2D Covering recent work with grad student
Rajasekhar Inkulu (CCCG conference paper)

• Input
• Covering polygons Q Q1, Q2 , ... , Qm
• Target polygons (or point-sets) P P1, P2 , ...
  , Pn
• Output
• Translations g g 1, g 2 , ... , g m such that

STATIC
NP-COMPLETE
12
New Covering Research Example 2 2D Channel
Assignment recent work with Masters student Sa
Liu (GLOBECOM conference paper)

• Input
• r x c cell grid
• q channels (frequencies)
• D(i,j) cell(i,j) traffic demand
• cell-phone calls
• Co-channel interference constraints based on
  interference threshold value
• Output
• Feasible assignment of channels to cells that
• satisfies demand and co-channel interference
  constraints
• uses minimum number of channels

STATIC
NP-COMPLETE
e.g. no channel repetition within any 2 x 2 square
13
Reduction to Establish Lower Bound

• Tightening a Lower Bound for Maximum-Area
  Axis-Parallel Rectangle Problem
• from W(n) to W(nlgn)

14
Approach

• Go around lower bound brick wall by
• examining strong lower bounds for some similar
  problems
• transforming a similar problem to our problem
• this process is similar to what we do when we
  prove problems NP-complete

worst-case bounds on problem
n5
2n
1
n2
n log2 n
n
15
Lower Bounds for Related Problems
LargestInnerRectangle W (n)
polygon
LargestInnerCircle Q (n log n)
point set
worst-case bounds on our problem
16
Lower Bound of W(n log n) by Transforming a
(seemingly unrelated) Problem
MAX-GAP instance given n real numbers x1, x2,
... xn find the maximum difference between 2
consecutive numbers in the sorted list.
O(n) time transformation
Rectangle area is a solution to the MAX-GAP
instance
specialized polygon
Rectangle algorithm must take as least as much
time as MAX-GAP. MAX-GAP is known to be in W(n
log n).
Rectangle algorithm must take W(n log n) time for
specialized polygons. Transforming yet another
different problem yields bound for unspecialized
polygons.
17
NP-Completeness
Chapter 34
18
Basic Concepts

• Polynomial Time Problem Formalization
• Polynomial-Time Verification
• NP-Completeness Reducibility
• NP-Completeness Proofs
• Expanding List of Hard Problems via Reduction

19
Polynomial Time Problem Formalization

• Formalize notions of abstract and concrete
  decision problems
• Encoding a problem
• Define class of polynomial-time solvable decision
  problems
• Define complexity class
• Express relationship between decision problems
  and algorithms that solve them using framework of
  formal language theory

20
Polynomial Time Problem Formalization
(continued)

• Solvable in Polynomial Time
• i.e. O(nk) for some positive constant k
• Tractable k is typically small (e.g. 1, 2, 3)
• Typically holds across computation models
• Closure properties (chaining/composition)
• Abstract Problem
• Decision Problem yes/no answer
• impose bound on optimization problem to recast

21
Polynomial Time Problem Formalization
(continued)

• Encoding of set S of abstract objects is mapping
  e from S to set of binary strings
• Concrete Problem instance set is set of binary
  strings
• Concrete Problem is Polynomial-Time Solvable if
  there exists an algorithm to solve it in time
  O(nk) for some positive constant k
• Complexity Class P set of polynomial-time
  solvable concrete decision problems
• Given abstract decision problem Q that maps
  instance set I to 0,1 can induce a
  concrete decision problem e(Q)
• If solution to abstract-problem instance
  
• then solution to concrete-problem instance
  is also Q(i)

22
Polynomial Time Problem Formalization
(continued)
Goal Extend definition of polynomial-time
solvability from concrete to abstract using
encodings as bridge

• is polynomial-time
  computable if there exists a polynomial-time
  algorithm A that, given any input
  , produces f(x) as output.
• For set I of problem instances, encodings are
  polynomially related if there exist 2
  polynomial-time computable functions f12, f21
  such that
• If 2 encodings e1, e2 of abstract problem are
  polynomially related, then polynomial-time
  solvability of problem is encoding-independent

23
Polynomial Time Problem Formalization
(continued)

• Formal Language Framework
• Alphabet S is finite set of symbols
• Language L over S is any set of strings
  consisting of symbols from S
• Set of instances for decision problem Q is
• View Q as language L over S 0,1
• Algorithm A accepts string if,
  given input x, output A(x) is 1
• Language accepted by A is set of strings A accepts

24
Polynomial Time Problem Formalization
(continued)

• Formal Language Framework (continued)
• Language L is decided by algorithm A if every
  binary string in L is accepted by A and every
  binary string not in L is rejected by A
• Language L is accepted in polynomial time by
  algorithm A if it is accepted by A and if exists
  constant k such that, for any length-n string
  , A accepts x in time O(nk)
• Language L is decidable in polynomial time by
  algorithm A if it is accepted by A and if exists
  constant k such that, for any length-n string
  , A correctly decides in time O(nk) if

25
Polynomial Time Problem Formalization
(continued)

• Formal Language Framework (continued)
• Complexity Class is a set of languages whose
  membership is determined by a complexity measure
  (i.e. running time) of algorithm that determines
  whether a given string belongs to a language.
• We already defined Complexity Class P set of
  polynomial-time solvable concrete decision
  problems
• Language Framework gives other definitions

26
Polynomial-Time Verification

• Examine algorithms that verify membership in
  languages
• Define class of decision problems whose solutions
  can be verified in polynomial time

27
Polynomial-Time Verification (continued)

• Verification Algorithms
• algorithm A verifies language L if for any string
  certificate y that A can use to prove
  that
• also, for any string certificate y that A can
  use to prove that
• language verified by verification algorithm A is

28
Polynomial-Time Verification (continued)

• The Complexity Class NP
• languages that can be verified by a
  polynomial-time algorithm
• A verifies language L in polynomial time
• The Complexity Class co-NP
• set of languages such that

29
Polynomial-Time Verification (continued)
P
P

P
34.3
source 91.503 textbook Cormen et al.
30
NP-Completeness Reducibility

• Polynomial-Time Reducibility
• Define complexity class of NP-complete problems
• Relationships among complexity classes
• Circuit satisfiability as an NP-complete problem

31
NP-Completeness Reducibility (continued)

• Q can be reduced to Q if any instance of Q can
  be easily rephrased as an instance of Q, the
  solution to which provides solution to instance
  of Q
• L1 is polynomial-time reducible to L2
  if exists
  polynomial-time computable function

34.4
source 91.503 textbook Cormen et al.
32
NP-Completeness Reducibility (continued)
34.5
34.3
source 91.503 textbook Cormen et al.
33
NP-Completeness Reducibility (continued)
Theorem - If any NP-complete problem is
polynomial-time solvable, then PNP. -
Equivalently, if any problem in NP is not
polynomial-time solvable, then no NP-complete
problem is polynomial-time solvable.
source 91.503 textbook Cormen et al.
34
NP-Completeness Reducibility (continued)
Circuit Satisfiability
circuit inputs

• Satisfying Assignment truth assignment inducing
  output 1
• GOAL Show Circuit Satisfiability is NP-complete

Language Def CIRCUIT-SAT ltCgtC is a
satisfiable boolean combinational circuit.
34.8
35
NP-Completeness Reducibility (continued)
Circuit Satisfiability

• CIRCUIT-SAT is in NP
• Construct 2-input polynomial-time algorithm A
  that verifies CIRCUIT-SAT
• Input 1 encoding of circuit C
• Input 2 certificate assignment of boolean
  values to all wires of C
• Algorithm A
• For each logic gate A checks if certificate value
  for gates output wire correctly computes gates
  function based on input wire values
• If output of entire circuit is 1, algorithm
  outputs 1
• Otherwise, algorithm outputs 0
• For satisfiable circuit, there is a
  (polynomial-length) certificate that causes A to
  output 1
• For unsatisfiable circuit, no certificate can
  cause A to output 1
• Algorithm A runs in polynomial time

source 91.503 textbook Cormen et al.
36
NP-Completeness Reducibility (continued)
Circuit Satisfiability

• CIRCUIT-SAT is NP-hard
• L represents some language in NP
• Create polynomial-time algorithm F computing
  reduction function f that maps every binary
  string x to a circuit Cf(x) such that
• Must exist algorithm A verifying L in polynomial
  time. (A from NP)
• F uses A to compute f.
• Represent computation of A as sequence of
  configurations
• F constructs single circuit that computes all
  configurations produced by initial configuration

source 91.503 textbook Cormen et al.
37
NP-Completeness Proofs (continued)

• Proving a Language NP-Complete
• Proving a Language NP-Hard
• all steps except (1)

source 91.503 textbook Cormen et al.
38
NP-Completeness Proofs (continued)

• Reducing Boolean Circuit Satisfiability to
  Boolean Formula Satisfiability
• Boolean Formula Satisfiability Instance of
  language SAT is a boolean formula f consisting
  of
• n boolean variables x1, x2, ... , xn
• m boolean connectives boolean function with 1 or
  2 inputs and 1 output
• e.g. AND, OR, NOT, implication, iff
• parentheses
• truth, satisfying assignments notions apply

source 91.503 textbook Cormen et al.
39
NP-Completeness Proofs (continued)

• Show SAT is NP-Complete
• SAT is in NP via argument similar to circuit case
• Reduce Boolean Circuit Satisfiability to Boolean
  Formula Satisfiability
• Careful! Naive approach might have shared
  subformulas and cause formula size to grow
  exponentially!

source 91.503 textbook Cormen et al.
40
NP-Completeness Proofs (continued)

• Show SAT is NP-Complete (continued)
• Reduce Boolean Circuit Satisfiability to Boolean
  Formula Satisfiability (continued)
• Approach For each wire xi in circuit C, formula
  f has variable xi. Express gate operation.

34.10
source 91.503 textbook Cormen et al.
41
NP-Completeness Proofs (continued)

• Reducing Formula Satisfiability to
  3-CNF-Satisfiability
• Boolean Formula Satisfiability Instance of
  language SAT is a boolean formula f
• CNF conjunctive normal form
• conjunction AND of clauses
• clause OR of literal(s)
• 3-CNF each clause has exactly 3 distinct
  literals
• Show 3-CNF is NP-Complete

source 91.503 textbook Cormen et al.
42
NP-Completeness Proofs (continued)

• Show 3-CNF is NP-Complete (continued)
• 3CNF is in NP via argument similar to circuit
  case
• Reduce Formula Satisfiability to
  3-CNF-Satisfiability
• 3 steps progressively transforming formula f
  closer to 3CNF
• 1) Similar to reduction

Binary ParseTree
34.11
source 91.503 textbook Cormen et al.
43
NP-Completeness Proofs (continued)

• Show 3-CNF is NP-Complete (continued)
• 3CNF is in NP via argument similar to circuit
  case
• Reduce Formula Satisfiability to
  3-CNF-Satisfiability
• 3 steps progressively transforming formula f
  closer to 3CNF
• 2) Convert each clause fi into conjunctive
  normal form
• construct truth table for fi
• using entries for 0, build DNF formula for
• convert DNF into CNF formula fi by applying
  DeMorgans laws
• each clause has at most 3 literals

source 91.503 textbook Cormen et al.
44
NP-Completeness Proofs (continued)

• Show 3-CNF is NP-Complete (continued)
• 3CNF is in NP via argument similar to circuit
  case
• Reduce Formula Satisfiability to
  3-CNF-Satisfiability
• 3 steps progressively transforming formula f
  closer to 3CNF
• 3) Convert each clause fi so it has exactly 3
  distinct literals
• add padding variables
• create fi
• Size of resulting formula is polynomial in length
  of original formula and reduction takes only
  polynomial time and 3-CNF formula

source 91.503 textbook Cormen et al.
45
Expanding List of Hard Problems via Reduction

• Relationships among some NP-complete problems

Circuit-SAT
source Garey Johnson
source 91.503 textbook Cormen et al.
46
Vertex Cover
A
B
vertex cover of size 2
Vertex Cover of an undirected graph G(V,E) is a
subset
F
47
Dominating Set
source Garey Johnson
48
Clique
Clique of an undirected graph G(V,E) is a
complete subgraph of G.
clique of size 3
source Garey Johnson
49
Hamiltonian Cycle
34.2
Hamiltonian Cycle of an undirected graph G(V,E)
is a simple cycle that contains each vertex in V.
source 91.503 textbook Cormen et al.
50
Reducing 3-CNF-SAT to Clique
source 91.503 textbook Cormen et al.
Construct graph G such that f is satisfiable iff
G has clique of size k
x1
x3
x1
x2
x2
34.14
51
Reducing Clique to VertexCover
Construct complement G (V, E ) of graph G such
that G has clique of size k iff graph G has a
vertex cover of size V-k.
34.15
source 91.503 textbook Cormen et al.
52
Literature
53
Resources

• Recommended
• Computers Intractability
• by Garey Johnson
• W.H.Freeman
• 1990
• ISBN 0716710455.

54
How to Treat
NP-Hard or Complete Problems
55
How to Treat NP-Complete Problems??

• Approximation Algorithms
• Heuristic Upper or Lower Bounds
• Greedy, Simulated Annealing, Genetic Alg, AI
• Mathematical Programming
• Linear Programming for part of problem
• Integer Programming
• Quadratic Programming...
• Search Space Exploration
• Gradient Descent, Local Search, Pruning,
  Subdivision
• Randomization, Derandomization
• Leverage/Impose Problem Structure
• Leverage Similar Problems