Loading...

PPT – Algorithms PowerPoint presentation | free to download - id: 554e0-ZDc1Z

The Adobe Flash plugin is needed to view this content

Algorithms

- NP-Complete Problems

Polynomial time algorithms

- Most algorithms we have studied so far are

polynomial time in the size of their inputs n. - Worst case running time is O(nk) for some

constant k. - Problems that are solvable by polynomial time

algorithms are said to be tractable.

Whats so great about polynomial time?

- 1. Although we would consider a problem that is

?(n100) intractable, this type of problem is

rarely encountered. The exponents on the

polynomial are usually small. - 2. For most reasonable models of computation, a

problem that can be solved in polynomial time in

one model can be solved in polynomial time in

another.

Closure properties of polynomial algorithms

- Closed under addition
- f(n) g(n)
- Closed under multiplication
- f(n)g(n)
- Closed under composition
- f(g(n))

Why is polynomial time an important goal?

- An algorithm is efficient (or polynomial time)

if there exists a polynomial p(n) such that the

algorithm can solve any instance of size n in

time O(p(n)). - A problem is tractable if it has an efficient

algorithm. - This is valid only if we avoid algorithms
- with extremely large constants
- with large values for k O(nk)
- with very large operands.

Do all problems have polynomial time solutions?

- NO!
- Some problems are not solvable
- Turings Halting problem is an example
- Cannot be solved by any computer no matter how

much time is given - Some problems can be shown to require

super-polynomial time to solve. - There is a a large class of important problems

for which we do not know the answer. These are

the NP complete problems.

Some Classifications

- UnsolvableTuring problem
- Solvable
- Provably intractable (infeasible)
- Probably intractable (NP-complete)
- Tractable (feasible)
- hard (gt n2)
- easy (most of the problems we have studied so far)

NP Completeness

- The Question!!
- Does P NP?
- This is an unanswered but very important

question. - What are P, NP, and NP-complete?
- Why is the question important?
- What do we think the answer is?
- What if the answer is what we think it is?

NP Complete Problems

- The complexity status of these problems is

unknown - There is no established lower bound
- But, it is known that if one of these problems

can be solved in polynomial time, all of them

can. - Very important equivalence class

What Do We Think?

- Most computer scientists believe that the NP

complete problems are intractable. - Why?
- Because there are a lot of these problems
- Many of them are very important problems
- People have been trying to find efficient

algorithms for them for a long time and no one

has succeeded.

Many P and NP-Complete Problems are Closely

Related

- Shortest path problem and longest path problem
- Euler tour and Hamiltonian Cycle
- Euler tour of connected, directed graph visits

each edge exactly once - Hamiltonian cycle begins and ends at same vertex

and visits each vertex exactly once

2-SAT and 3-SAT

- 2-Conjunctive Normal Form (CNF) Satisfiability
- 3-Conjunctive Normal Form (CNF) Satisfiability

The Satisfiability Problem

- Given a Boolean formula containing variables

whose value are 0 or 1, connected by the Boolean

connectives ?, ?, and ?. - A Boolean formula is satisfiable if there is an

assignment of values to variables that causes the

formula to evaluate to 1

k-CNF

- Informal definition A formula is in

k-conjunctive normal form if it is the AND of

clauses of ORs of exactly k variables or their

negations. - Example of 2-CNF formula
- (x1 ? ?x2) ? (?x1 ? x3) ? (?x2 ? ?x3)
- What is a satisfying assignment of values to

variables?

CNF-Satisfiability

- A polynomial time algorithm exists for to

determine if a 2-CNF formula is satisfiable. - 3-CNF is NP-complete

Three Classes of Problems

- P (Polynomial)
- Problems solvable in polynomial time
- Can be solved in time O(nk) where k is a constant
- NP (Non-deterministic Polynomial)
- Problems that are verifiable in polynomial time
- Given a certificate of a solution, we can

verify that the solution is correct in polynomial

time - For Hamiltonian path the certificate is a

sequence of vertices - For 3-SAT, the certificate is an assignment of

values to variables - NPC (NP Complete)

NP Complete

- Informal definition A problem is in the class

NPC if it is in NP and is as hard as any

problem in NP. - If any problem in the class NP can be solved in

polynomial time, then all can be solved in

polynomial time

Proofs of NP-Completeness

- Different from other proofs we have done
- Rather than show that a problem has an efficient

algorithm, we will be demonstrating that it is

hard.

Theory of NP-Completeness

- Limited to one type of problem - decision

problems - Each element of S (the set of solutions) is an

element of yes,no (or 0,1) - Seems like a severe restriction, but it is not
- Usually dealing with optimization problems that

are easily restated as a decision problem. - Can show that if the decision problem is easy,

then the optimization problem is easy.

Optimization Problem vs Decision Problem

- Shortest path problem as an optimization problem
- Given a graph G (V,E), two vertices u,v ? V,

what is the shortest path that exists in G

between u and v? - Shortest path problem as a decision problem
- Given a graph G (V,E), two vertices u,v ? V and

a nonnegative integer k, does a path exist in G

between u and v whose length is at most k?

General Approach

- To change an optimization problem to a decision

problem, we can usually just impose a bound on

the value to be optimized. - minimization problem example
- length of shortest path at most k
- maximization problem example
- length of longest path at least k

General statements

- Usually, if we can solve the optimization problem

quickly, we can solve the decision problem

quickly. - Also, if we can provide evidence that the

decision problem is hard, we also provide

evidence that the optimization problem is hard.

The Class P

- The complexity class P is the set of concrete

decision problems that are solvable in polynomial

time.

Nondeterministic Polynomial (NP)

- Let an oracle guess an answer to the problem

(certificate). - If we can find an algorithm that can verify that

the answer is correct in polynomial time, then

this problem is in the class NP - P is clearly a subset of NP

P NP?

NP

NP

NP-complete

NP

NP-complete

P

P

Example HAM-CYCLE

- Hamiltonian Cycle Problem
- A Hamiltonian cycle of an undirected graph

G(V,E) is a simple cycle that contains each

vertex in V. - The Hamiltonian-cycle problem asks if a given

graph contains a Hamiltonian cycle.

HAM-CYCLE ??NP

- HAM-CYCLE is an element of NP because we can

verify an answer - Given a list of vertices, does the list represent

a Hamiltonian cycle? - We can find a polynomial time algorithm to

determine if this list is a Hamiltonian cycle. - Is the first vertex the same as the last vertex?
- Are all vertices visited?
- For each vertex in the list, is there an edge to

the next vertex in the list? - Is any vertex other than the first repeated

Showing Problems to be NP- Complete

- Restate problem as a decision problem
- Demonstrate that the decision problem is in the

class NP - Show that a problem known to be NP-Complete can

be reduced to the current problem in polynomial

time.

NP-Completeness and Reducibility

- NP-complete problems
- informally they are the hardest problems in the

class NP - no polynomial time algorithm is known to solve

any of them - but every NP-complete problem is reducible to

every other NP-complete problem in polynomial time

Reducibility

Polynomial time

Set of instances of Problem A

Set of instances of Problem B

Decision problem A is polynomial-time reducible

to decision problem B if a polynomial time

algorithm can be developed which changes each

instance of problem A to an instance of problem B

such that if the answer for an instance of B is

yes, the answer to the corresponding instance of

A is yes.

Algorithm from Reduction for Solving Problem A

YES

Polynomial-time reduction algorithm

Polynomial-time algorithm to decide B

?

?

NO

A First NP-Complete Problem

- This technique for showing that a problem is in

the class NP-Complete requires that we have one

NP-Complete problem to begin with - Circuit satisifiability was the first problem to

be shown to be in NP-Complete.

Lemma 34.3

- If Q1, Q2 are problems such that
- Q1 ?p Q2, then Q2 ? P implies Q1 ?P.
- Proof
- Suppose A1 is a polynomial time algorithm that

reduces instances of Q1to instances of Q2 and

that A2 is a polynomial time algorithm for

problem Q2. Then the concatenation of A1 and A2

is a polynomial time algorithm for Q1.

An Instance of Problem Q1

Reduction (polynomial)

An Instance of Problem Q2

Polynomial algorithm for Q1

Polynomial algorithm for Q2

Solution for any Instance (YES,NO)

NP-Complete

- A problem that satisfies property 2, but not

necessarily property 1 is NP-hard.

Theorem 34.4

- If any NP-complete problem is polynomial-time

solvable, then P NP. - If any problem in NP is not polynomial-time

solvable, then all NP-complete problems are not

polynomial-time solvable.

Proof of Theorem 34.4

- Proof of Statement 1 If any NP-complete problem

is polynomial-time solvable, then P NP. - Suppose that Q ??P and Q ??NPC.
- Then, for any Q ??NP, we know that Q ?pQ by

property 2 of NP-completeness. By Lemma 34.3,

Q ??P.

P

Q

NPC

Q

NP

Q

P/NP

NPC

Proof of Theorem 34.4

- Proof of Statement 2 If any problem in NP is not

polynomial-time solvable, then all NP-complete

problems are not polynomial-time solvable. - Suppose that there exists an Q ??NP such that Q

?P. Let Q ??NPC be any NP-complete problem and

suppose that Q ? P. - By Lemma 34.3, Q ?pQ and so Q ?P. Contradiction

NPC

Q

P

Q

NPC

Importance of Theorem

- The theorem make the focus of the PNP question

the NP-complete problems - Most theoretical computer scientists believe that

P ??NP. - But, if a polynomial time algorithm exists for

any NPC problem, then P NP. - But, no one has been able to do this so far.

NP-Completeness Proofs

- Suppose that we can show that one problem (Q) is

in the class NP-complete. Then we can show that

another problem Q is in the class NP-complete by

showing that - Q is in NP
- Q can be reduced to Q in polynomial time.
- Describe an algorithm for the reduction
- Prove that the answer to Q is yes iff the answer

to Q is yes - Prove that the algorithm for the reduction runs

in polynomial time

Circuit Satisifiability

- Cook proved that the CIRCUIT-SAT problem is

NP-complete. - 3-SAT was also shown to be NPC by Cook.
- It is quite difficult to show that a particular

problem in NP can be reduced to every other

problem in NP. - But once one problem is know to be NP-complete,

the others are easier. - Many other problems have since been proven to be

NP-complete.

Sketch of NP-Completeness Proof for CIRCUIT-SAT

- Very long and difficult in the details.
- Used to provide the basis for most other proofs

of NP-Completeness. - Problem domain is combinatorial circuits.

Combinatorial Circuit Components

Circuit-Satisfiability Problem

- Given a boolena combinatorial circuit composed of

AND, OR, and NOT gates, is it satisfiable?

Demonstrating Circuit Satisfiability is NP

Complete

- This proof was done from first principles.
- It did not depend on the existence of any other

NPC problems. - A problem Q is NP-Complete if
- 1. It is an element of the class NP
- 2. Q ?pQ for every Q in NP

Step 1 Prove CIRCUIT-SAT ? NP

- Develop an algorithm A that can verify a solution

in polynomial time. - Inputs for A
- A standard representation of a combinatorial

circuit - An assignment of input values to the circuit
- Algorithm A determines the output of the

combinatorial circuit. Output is 0 or 1. - This can be done in polynomial time (linear for a

clever algorithm.) - Therefore CIRCUIT-SAT ?NP

Step 2 Reduction

- We need to show that every other problem in NP

can be reduced to CIRCUIT-SAT. - The basic argument is
- A combinatorial circuit can be used to implement

a computer - Program
- Memory
- Program counter
- Working storage, etc.
- You can show that every problem in NP can be

mapped onto operations that can be represented as

a sequences of states of combinatorial circuits. - You show that this can be done in polynomial

time. - It is not simple to show all of this, but in the

end you can show that any problem in NP can be

reduced to CIRCUIT-SAT in polynomial time

Implications of CIRCUIT-SAT?NPC

- CIRCUIT-SAT is the seed problem in NPC
- Once we know that one problem is in NPC, we can

use it to demonstrate that other problems are in

NPC using a simpler procedure.

CIRCUIT-SAT

SAT

3-SAT

CLIQUE

SUBSET-SUM

VERTEX-COVER

HAM-CYCLE

TSP

EXAMPLE

- Suppose someone has proven that the problem of

determining if a graph G has a Hamiltonian cycle

has been shown to be NP-complete. - And we want to show that the Traveling Salesman

Problem (TSP) is NP-complete. - Two steps
- show that TSP is in the class NP
- show that HAM reduces to TSP in polynomial time

Traveling Salesman Problem

- TSP Optimization Problem
- A salesman must visit n cities. He must visit

each city exactly one time and finish with the

city at which he starts. Model possible routes

as an undirected graph with weights on the edges.

- c(I,j) is cost of traveling from city I to j
- TSP Decision Problem
- Change it to a decision problem by making the

problem to find a route of length at least k.

Step 1

- Show that TSP is in the class NP.
- If we are given a set of n1 vertices that we

claim constitute a tour of length at most k, then

we can - check to see that each vertex occurs exactly one

time - check to see that beginning and ending vertices

are the same - check to see that edges exist between adjacent

vertices in the graph - see if the sum of the edges is ? k

Step 1 continued

- We can show that each of these steps can be done

in polynomial time - Therefore TSP is an element of NP

Step 2

- Now we need to show that any HAM problem can be

reduced to a TSP problem.

Reduction

HAM

TSP

Problem we want to show to be NPC

Known NPC Problem

The Two Problems

- HAM
- A Hamiltonian cycle of an undirected graph

G(V,E) is a simple cycle that contains each

vertex in V. - The Hamiltonian-cycle problem asks if a given

graph contains a Hamiltonian cycle. - TSP
- A TSP circuit in of a undirected graph with edge

weights visits each vertex exactly one time and

begins and ends at the same vertex. - The TSP problem asks if a given graph contains a

TSP circuit of length less than k.

The Reduction from A to B

- Consider an instance of problem A. Show how that

instance can be changed to an instance of B. - Prove that the answer to B is yes iff the answer

to Q is yes - Prove that the algorithm for the reduction runs

in polynomial time.

The Reduction Creating an Instance

- Now we need to show that any HAM problem can be

reduced to a TSP problem. - Let G(V,E) be an instance of HAM
- Construct an instance of TSP as follows
- Form the complete graph G(V,E)
- Define the cost of the edges as
- c(i,j) 0 if (i,j) is in E, 1 otherwise
- We look for a tour of length 0

G

G

TSP

HAM

Step 2 continued

- Now we must show that graph G has a Hamiltonian

cycle iff G has a tour of cost at most 0. - G has Hamiltonian cycle implies G has tour of

cost at most n. - Suppose that graph G has a Hamiltonian cycle h.
- Each edge in h belongs to E and thus there is an

edge from vi to vi1 with a cost of 0 in G - Thus, h is a tour in G with a cost of 0.

Step 2 continued

- G has a tour of cost at most 0 implies G has as

Hamiltonian cycle. - Suppose that G has a tour h of cost at most 0.
- Since the cost of the edges is 0 or 1, none of

the edges can have a cost of 1if the cost of the

tour is at most 0 - Therefore all edges are in E.
- So h visits each vertex exactly once, contains

only edges in E, and is therefore a Hamiltonian

cycle in G.