# Algorithms - PowerPoint PPT Presentation

PPT – Algorithms PowerPoint presentation | free to download - id: 554e0-ZDc1Z The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Algorithms

Description:

### ... to find efficient algorithms for them for a long time and no one has succeeded. ... Now we need to show that any HAM problem can be reduced to a TSP problem. ... – PowerPoint PPT presentation

Number of Views:17
Avg rating:3.0/5.0
Slides: 61
Provided by: susanb69
Category:
Tags:
Transcript and Presenter's Notes

Title: Algorithms

1
Algorithms
• NP-Complete Problems

2
Polynomial time algorithms
• Most algorithms we have studied so far are
polynomial time in the size of their inputs n.
• Worst case running time is O(nk) for some
constant k.
• Problems that are solvable by polynomial time
algorithms are said to be tractable.

3
Whats so great about polynomial time?
• 1. Although we would consider a problem that is
?(n100) intractable, this type of problem is
rarely encountered. The exponents on the
polynomial are usually small.
• 2. For most reasonable models of computation, a
problem that can be solved in polynomial time in
one model can be solved in polynomial time in
another.

4
Closure properties of polynomial algorithms
• f(n) g(n)
• Closed under multiplication
• f(n)g(n)
• Closed under composition
• f(g(n))

5
Why is polynomial time an important goal?
• An algorithm is efficient (or polynomial time)
if there exists a polynomial p(n) such that the
algorithm can solve any instance of size n in
time O(p(n)).
• A problem is tractable if it has an efficient
algorithm.
• This is valid only if we avoid algorithms
• with extremely large constants
• with large values for k O(nk)
• with very large operands.

6
Do all problems have polynomial time solutions?
• NO!
• Some problems are not solvable
• Turings Halting problem is an example
• Cannot be solved by any computer no matter how
much time is given
• Some problems can be shown to require
super-polynomial time to solve.
• There is a a large class of important problems
for which we do not know the answer. These are
the NP complete problems.

7
Some Classifications
• UnsolvableTuring problem
• Solvable
• Provably intractable (infeasible)
• Probably intractable (NP-complete)
• Tractable (feasible)
• hard (gt n2)
• easy (most of the problems we have studied so far)

8
NP Completeness
• The Question!!
• Does P NP?
• This is an unanswered but very important
question.
• What are P, NP, and NP-complete?
• Why is the question important?
• What do we think the answer is?
• What if the answer is what we think it is?

9
NP Complete Problems
• The complexity status of these problems is
unknown
• There is no established lower bound
• But, it is known that if one of these problems
can be solved in polynomial time, all of them
can.
• Very important equivalence class

10
What Do We Think?
• Most computer scientists believe that the NP
complete problems are intractable.
• Why?
• Because there are a lot of these problems
• Many of them are very important problems
• People have been trying to find efficient
algorithms for them for a long time and no one
has succeeded.

11
Many P and NP-Complete Problems are Closely
Related
• Shortest path problem and longest path problem
• Euler tour and Hamiltonian Cycle
• Euler tour of connected, directed graph visits
each edge exactly once
• Hamiltonian cycle begins and ends at same vertex
and visits each vertex exactly once

12
2-SAT and 3-SAT
• 2-Conjunctive Normal Form (CNF) Satisfiability
• 3-Conjunctive Normal Form (CNF) Satisfiability

13
The Satisfiability Problem
• Given a Boolean formula containing variables
whose value are 0 or 1, connected by the Boolean
connectives ?, ?, and ?.
• A Boolean formula is satisfiable if there is an
assignment of values to variables that causes the
formula to evaluate to 1

14
k-CNF
• Informal definition A formula is in
k-conjunctive normal form if it is the AND of
clauses of ORs of exactly k variables or their
negations.
• Example of 2-CNF formula
• (x1 ? ?x2) ? (?x1 ? x3) ? (?x2 ? ?x3)
• What is a satisfying assignment of values to
variables?

15
CNF-Satisfiability
• A polynomial time algorithm exists for to
determine if a 2-CNF formula is satisfiable.
• 3-CNF is NP-complete

16
Three Classes of Problems
• P (Polynomial)
• Problems solvable in polynomial time
• Can be solved in time O(nk) where k is a constant
• NP (Non-deterministic Polynomial)
• Problems that are verifiable in polynomial time
• Given a certificate of a solution, we can
verify that the solution is correct in polynomial
time
• For Hamiltonian path the certificate is a
sequence of vertices
• For 3-SAT, the certificate is an assignment of
values to variables
• NPC (NP Complete)

17
NP Complete
• Informal definition A problem is in the class
NPC if it is in NP and is as hard as any
problem in NP.
• If any problem in the class NP can be solved in
polynomial time, then all can be solved in
polynomial time

18
Proofs of NP-Completeness
• Different from other proofs we have done
• Rather than show that a problem has an efficient
algorithm, we will be demonstrating that it is
hard.

19
Theory of NP-Completeness
• Limited to one type of problem - decision
problems
• Each element of S (the set of solutions) is an
element of yes,no (or 0,1)
• Seems like a severe restriction, but it is not
• Usually dealing with optimization problems that
are easily restated as a decision problem.
• Can show that if the decision problem is easy,
then the optimization problem is easy.

20
Optimization Problem vs Decision Problem
• Shortest path problem as an optimization problem
• Given a graph G (V,E), two vertices u,v ? V,
what is the shortest path that exists in G
between u and v?
• Shortest path problem as a decision problem
• Given a graph G (V,E), two vertices u,v ? V and
a nonnegative integer k, does a path exist in G
between u and v whose length is at most k?

21
General Approach
• To change an optimization problem to a decision
problem, we can usually just impose a bound on
the value to be optimized.
• minimization problem example
• length of shortest path at most k
• maximization problem example
• length of longest path at least k

22
General statements
• Usually, if we can solve the optimization problem
quickly, we can solve the decision problem
quickly.
• Also, if we can provide evidence that the
decision problem is hard, we also provide
evidence that the optimization problem is hard.

23
The Class P
• The complexity class P is the set of concrete
decision problems that are solvable in polynomial
time.

24
Nondeterministic Polynomial (NP)
• Let an oracle guess an answer to the problem
(certificate).
• If we can find an algorithm that can verify that
the answer is correct in polynomial time, then
this problem is in the class NP
• P is clearly a subset of NP

25
P NP?
NP
NP
NP-complete
NP
NP-complete
P
P
26
Example HAM-CYCLE
• Hamiltonian Cycle Problem
• A Hamiltonian cycle of an undirected graph
G(V,E) is a simple cycle that contains each
vertex in V.
• The Hamiltonian-cycle problem asks if a given
graph contains a Hamiltonian cycle.

27
HAM-CYCLE ??NP
• HAM-CYCLE is an element of NP because we can
• Given a list of vertices, does the list represent
a Hamiltonian cycle?
• We can find a polynomial time algorithm to
determine if this list is a Hamiltonian cycle.
• Is the first vertex the same as the last vertex?
• Are all vertices visited?
• For each vertex in the list, is there an edge to
the next vertex in the list?
• Is any vertex other than the first repeated

28
Showing Problems to be NP- Complete
• Restate problem as a decision problem
• Demonstrate that the decision problem is in the
class NP
• Show that a problem known to be NP-Complete can
be reduced to the current problem in polynomial
time.

29
NP-Completeness and Reducibility
• NP-complete problems
• informally they are the hardest problems in the
class NP
• no polynomial time algorithm is known to solve
any of them
• but every NP-complete problem is reducible to
every other NP-complete problem in polynomial time

30
Reducibility
Polynomial time
Set of instances of Problem A
Set of instances of Problem B
Decision problem A is polynomial-time reducible
to decision problem B if a polynomial time
algorithm can be developed which changes each
instance of problem A to an instance of problem B
such that if the answer for an instance of B is
yes, the answer to the corresponding instance of
A is yes.
31
Algorithm from Reduction for Solving Problem A
YES
Polynomial-time reduction algorithm
Polynomial-time algorithm to decide B
?
?
NO
32
A First NP-Complete Problem
• This technique for showing that a problem is in
the class NP-Complete requires that we have one
NP-Complete problem to begin with
• Circuit satisifiability was the first problem to
be shown to be in NP-Complete.

33
Lemma 34.3
• If Q1, Q2 are problems such that
• Q1 ?p Q2, then Q2 ? P implies Q1 ?P.
• Proof
• Suppose A1 is a polynomial time algorithm that
reduces instances of Q1to instances of Q2 and
that A2 is a polynomial time algorithm for
problem Q2. Then the concatenation of A1 and A2
is a polynomial time algorithm for Q1.

34
An Instance of Problem Q1
Reduction (polynomial)
An Instance of Problem Q2
Polynomial algorithm for Q1
Polynomial algorithm for Q2
Solution for any Instance (YES,NO)
35
NP-Complete
• A problem that satisfies property 2, but not
necessarily property 1 is NP-hard.

36
Theorem 34.4
• If any NP-complete problem is polynomial-time
solvable, then P NP.
• If any problem in NP is not polynomial-time
solvable, then all NP-complete problems are not
polynomial-time solvable.

37
Proof of Theorem 34.4
• Proof of Statement 1 If any NP-complete problem
is polynomial-time solvable, then P NP.
• Suppose that Q ??P and Q ??NPC.
• Then, for any Q ??NP, we know that Q ?pQ by
property 2 of NP-completeness. By Lemma 34.3,
Q ??P.

P
Q
NPC
Q
NP
Q
P/NP
NPC
38
Proof of Theorem 34.4
• Proof of Statement 2 If any problem in NP is not
polynomial-time solvable, then all NP-complete
problems are not polynomial-time solvable.
• Suppose that there exists an Q ??NP such that Q
?P. Let Q ??NPC be any NP-complete problem and
suppose that Q ? P.
• By Lemma 34.3, Q ?pQ and so Q ?P. Contradiction

NPC
Q
P
Q
NPC
39
Importance of Theorem
• The theorem make the focus of the PNP question
the NP-complete problems
• Most theoretical computer scientists believe that
P ??NP.
• But, if a polynomial time algorithm exists for
any NPC problem, then P NP.
• But, no one has been able to do this so far.

40
NP-Completeness Proofs
• Suppose that we can show that one problem (Q) is
in the class NP-complete. Then we can show that
another problem Q is in the class NP-complete by
showing that
• Q is in NP
• Q can be reduced to Q in polynomial time.
• Describe an algorithm for the reduction
• Prove that the answer to Q is yes iff the answer
to Q is yes
• Prove that the algorithm for the reduction runs
in polynomial time

41
Circuit Satisifiability
• Cook proved that the CIRCUIT-SAT problem is
NP-complete.
• 3-SAT was also shown to be NPC by Cook.
• It is quite difficult to show that a particular
problem in NP can be reduced to every other
problem in NP.
• But once one problem is know to be NP-complete,
the others are easier.
• Many other problems have since been proven to be
NP-complete.

42
Sketch of NP-Completeness Proof for CIRCUIT-SAT
• Very long and difficult in the details.
• Used to provide the basis for most other proofs
of NP-Completeness.
• Problem domain is combinatorial circuits.

43
Combinatorial Circuit Components
44
Circuit-Satisfiability Problem
• Given a boolena combinatorial circuit composed of
AND, OR, and NOT gates, is it satisfiable?

45
Demonstrating Circuit Satisfiability is NP
Complete
• This proof was done from first principles.
• It did not depend on the existence of any other
NPC problems.
• A problem Q is NP-Complete if
• 1. It is an element of the class NP
• 2. Q ?pQ for every Q in NP

46
Step 1 Prove CIRCUIT-SAT ? NP
• Develop an algorithm A that can verify a solution
in polynomial time.
• Inputs for A
• A standard representation of a combinatorial
circuit
• An assignment of input values to the circuit
• Algorithm A determines the output of the
combinatorial circuit. Output is 0 or 1.
• This can be done in polynomial time (linear for a
clever algorithm.)
• Therefore CIRCUIT-SAT ?NP

47
Step 2 Reduction
• We need to show that every other problem in NP
can be reduced to CIRCUIT-SAT.
• The basic argument is
• A combinatorial circuit can be used to implement
a computer
• Program
• Memory
• Program counter
• Working storage, etc.
• You can show that every problem in NP can be
mapped onto operations that can be represented as
a sequences of states of combinatorial circuits.
• You show that this can be done in polynomial
time.
• It is not simple to show all of this, but in the
end you can show that any problem in NP can be
reduced to CIRCUIT-SAT in polynomial time

48
Implications of CIRCUIT-SAT?NPC
• CIRCUIT-SAT is the seed problem in NPC
• Once we know that one problem is in NPC, we can
use it to demonstrate that other problems are in
NPC using a simpler procedure.

49
CIRCUIT-SAT
SAT
3-SAT
CLIQUE
SUBSET-SUM
VERTEX-COVER
HAM-CYCLE
TSP
50
EXAMPLE
• Suppose someone has proven that the problem of
determining if a graph G has a Hamiltonian cycle
has been shown to be NP-complete.
• And we want to show that the Traveling Salesman
Problem (TSP) is NP-complete.
• Two steps
• show that TSP is in the class NP
• show that HAM reduces to TSP in polynomial time

51
Traveling Salesman Problem
• TSP Optimization Problem
• A salesman must visit n cities. He must visit
each city exactly one time and finish with the
city at which he starts. Model possible routes
as an undirected graph with weights on the edges.
• c(I,j) is cost of traveling from city I to j
• TSP Decision Problem
• Change it to a decision problem by making the
problem to find a route of length at least k.

52
Step 1
• Show that TSP is in the class NP.
• If we are given a set of n1 vertices that we
claim constitute a tour of length at most k, then
we can
• check to see that each vertex occurs exactly one
time
• check to see that beginning and ending vertices
are the same
• check to see that edges exist between adjacent
vertices in the graph
• see if the sum of the edges is ? k

53
Step 1 continued
• We can show that each of these steps can be done
in polynomial time
• Therefore TSP is an element of NP

54
Step 2
• Now we need to show that any HAM problem can be
reduced to a TSP problem.

Reduction
HAM
TSP
Problem we want to show to be NPC
Known NPC Problem
55
The Two Problems
• HAM
• A Hamiltonian cycle of an undirected graph
G(V,E) is a simple cycle that contains each
vertex in V.
• The Hamiltonian-cycle problem asks if a given
graph contains a Hamiltonian cycle.
• TSP
• A TSP circuit in of a undirected graph with edge
weights visits each vertex exactly one time and
begins and ends at the same vertex.
• The TSP problem asks if a given graph contains a
TSP circuit of length less than k.

56
The Reduction from A to B
• Consider an instance of problem A. Show how that
instance can be changed to an instance of B.
• Prove that the answer to B is yes iff the answer
to Q is yes
• Prove that the algorithm for the reduction runs
in polynomial time.

57
The Reduction Creating an Instance
• Now we need to show that any HAM problem can be
reduced to a TSP problem.
• Let G(V,E) be an instance of HAM
• Construct an instance of TSP as follows
• Form the complete graph G(V,E)
• Define the cost of the edges as
• c(i,j) 0 if (i,j) is in E, 1 otherwise
• We look for a tour of length 0

58
G
G
TSP
HAM
59
Step 2 continued
• Now we must show that graph G has a Hamiltonian
cycle iff G has a tour of cost at most 0.
• G has Hamiltonian cycle implies G has tour of
cost at most n.
• Suppose that graph G has a Hamiltonian cycle h.
• Each edge in h belongs to E and thus there is an
edge from vi to vi1 with a cost of 0 in G
• Thus, h is a tour in G with a cost of 0.

60
Step 2 continued
• G has a tour of cost at most 0 implies G has as
Hamiltonian cycle.
• Suppose that G has a tour h of cost at most 0.
• Since the cost of the edges is 0 or 1, none of
the edges can have a cost of 1if the cost of the
tour is at most 0
• Therefore all edges are in E.
• So h visits each vertex exactly once, contains
only edges in E, and is therefore a Hamiltonian
cycle in G.