Algorithms - PowerPoint PPT Presentation

Loading...

PPT – Algorithms PowerPoint presentation | free to download - id: 554e0-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Algorithms

Description:

... to find efficient algorithms for them for a long time and no one has succeeded. ... Now we need to show that any HAM problem can be reduced to a TSP problem. ... – PowerPoint PPT presentation

Number of Views:17
Avg rating:3.0/5.0
Slides: 61
Provided by: susanb69
Category:
Tags: algorithms | cook | do | ham | how | long

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Algorithms


1
Algorithms
  • NP-Complete Problems

2
Polynomial time algorithms
  • Most algorithms we have studied so far are
    polynomial time in the size of their inputs n.
  • Worst case running time is O(nk) for some
    constant k.
  • Problems that are solvable by polynomial time
    algorithms are said to be tractable.

3
Whats so great about polynomial time?
  • 1. Although we would consider a problem that is
    ?(n100) intractable, this type of problem is
    rarely encountered. The exponents on the
    polynomial are usually small.
  • 2. For most reasonable models of computation, a
    problem that can be solved in polynomial time in
    one model can be solved in polynomial time in
    another.

4
Closure properties of polynomial algorithms
  • Closed under addition
  • f(n) g(n)
  • Closed under multiplication
  • f(n)g(n)
  • Closed under composition
  • f(g(n))

5
Why is polynomial time an important goal?
  • An algorithm is efficient (or polynomial time)
    if there exists a polynomial p(n) such that the
    algorithm can solve any instance of size n in
    time O(p(n)).
  • A problem is tractable if it has an efficient
    algorithm.
  • This is valid only if we avoid algorithms
  • with extremely large constants
  • with large values for k O(nk)
  • with very large operands.

6
Do all problems have polynomial time solutions?
  • NO!
  • Some problems are not solvable
  • Turings Halting problem is an example
  • Cannot be solved by any computer no matter how
    much time is given
  • Some problems can be shown to require
    super-polynomial time to solve.
  • There is a a large class of important problems
    for which we do not know the answer. These are
    the NP complete problems.

7
Some Classifications
  • UnsolvableTuring problem
  • Solvable
  • Provably intractable (infeasible)
  • Probably intractable (NP-complete)
  • Tractable (feasible)
  • hard (gt n2)
  • easy (most of the problems we have studied so far)

8
NP Completeness
  • The Question!!
  • Does P NP?
  • This is an unanswered but very important
    question.
  • What are P, NP, and NP-complete?
  • Why is the question important?
  • What do we think the answer is?
  • What if the answer is what we think it is?

9
NP Complete Problems
  • The complexity status of these problems is
    unknown
  • There is no established lower bound
  • But, it is known that if one of these problems
    can be solved in polynomial time, all of them
    can.
  • Very important equivalence class

10
What Do We Think?
  • Most computer scientists believe that the NP
    complete problems are intractable.
  • Why?
  • Because there are a lot of these problems
  • Many of them are very important problems
  • People have been trying to find efficient
    algorithms for them for a long time and no one
    has succeeded.

11
Many P and NP-Complete Problems are Closely
Related
  • Shortest path problem and longest path problem
  • Euler tour and Hamiltonian Cycle
  • Euler tour of connected, directed graph visits
    each edge exactly once
  • Hamiltonian cycle begins and ends at same vertex
    and visits each vertex exactly once

12
2-SAT and 3-SAT
  • 2-Conjunctive Normal Form (CNF) Satisfiability
  • 3-Conjunctive Normal Form (CNF) Satisfiability

13
The Satisfiability Problem
  • Given a Boolean formula containing variables
    whose value are 0 or 1, connected by the Boolean
    connectives ?, ?, and ?.
  • A Boolean formula is satisfiable if there is an
    assignment of values to variables that causes the
    formula to evaluate to 1

14
k-CNF
  • Informal definition A formula is in
    k-conjunctive normal form if it is the AND of
    clauses of ORs of exactly k variables or their
    negations.
  • Example of 2-CNF formula
  • (x1 ? ?x2) ? (?x1 ? x3) ? (?x2 ? ?x3)
  • What is a satisfying assignment of values to
    variables?

15
CNF-Satisfiability
  • A polynomial time algorithm exists for to
    determine if a 2-CNF formula is satisfiable.
  • 3-CNF is NP-complete

16
Three Classes of Problems
  • P (Polynomial)
  • Problems solvable in polynomial time
  • Can be solved in time O(nk) where k is a constant
  • NP (Non-deterministic Polynomial)
  • Problems that are verifiable in polynomial time
  • Given a certificate of a solution, we can
    verify that the solution is correct in polynomial
    time
  • For Hamiltonian path the certificate is a
    sequence of vertices
  • For 3-SAT, the certificate is an assignment of
    values to variables
  • NPC (NP Complete)

17
NP Complete
  • Informal definition A problem is in the class
    NPC if it is in NP and is as hard as any
    problem in NP.
  • If any problem in the class NP can be solved in
    polynomial time, then all can be solved in
    polynomial time

18
Proofs of NP-Completeness
  • Different from other proofs we have done
  • Rather than show that a problem has an efficient
    algorithm, we will be demonstrating that it is
    hard.

19
Theory of NP-Completeness
  • Limited to one type of problem - decision
    problems
  • Each element of S (the set of solutions) is an
    element of yes,no (or 0,1)
  • Seems like a severe restriction, but it is not
  • Usually dealing with optimization problems that
    are easily restated as a decision problem.
  • Can show that if the decision problem is easy,
    then the optimization problem is easy.

20
Optimization Problem vs Decision Problem
  • Shortest path problem as an optimization problem
  • Given a graph G (V,E), two vertices u,v ? V,
    what is the shortest path that exists in G
    between u and v?
  • Shortest path problem as a decision problem
  • Given a graph G (V,E), two vertices u,v ? V and
    a nonnegative integer k, does a path exist in G
    between u and v whose length is at most k?

21
General Approach
  • To change an optimization problem to a decision
    problem, we can usually just impose a bound on
    the value to be optimized.
  • minimization problem example
  • length of shortest path at most k
  • maximization problem example
  • length of longest path at least k

22
General statements
  • Usually, if we can solve the optimization problem
    quickly, we can solve the decision problem
    quickly.
  • Also, if we can provide evidence that the
    decision problem is hard, we also provide
    evidence that the optimization problem is hard.

23
The Class P
  • The complexity class P is the set of concrete
    decision problems that are solvable in polynomial
    time.

24
Nondeterministic Polynomial (NP)
  • Let an oracle guess an answer to the problem
    (certificate).
  • If we can find an algorithm that can verify that
    the answer is correct in polynomial time, then
    this problem is in the class NP
  • P is clearly a subset of NP

25
P NP?
NP
NP
NP-complete
NP
NP-complete
P
P
26
Example HAM-CYCLE
  • Hamiltonian Cycle Problem
  • A Hamiltonian cycle of an undirected graph
    G(V,E) is a simple cycle that contains each
    vertex in V.
  • The Hamiltonian-cycle problem asks if a given
    graph contains a Hamiltonian cycle.

27
HAM-CYCLE ??NP
  • HAM-CYCLE is an element of NP because we can
    verify an answer
  • Given a list of vertices, does the list represent
    a Hamiltonian cycle?
  • We can find a polynomial time algorithm to
    determine if this list is a Hamiltonian cycle.
  • Is the first vertex the same as the last vertex?
  • Are all vertices visited?
  • For each vertex in the list, is there an edge to
    the next vertex in the list?
  • Is any vertex other than the first repeated

28
Showing Problems to be NP- Complete
  • Restate problem as a decision problem
  • Demonstrate that the decision problem is in the
    class NP
  • Show that a problem known to be NP-Complete can
    be reduced to the current problem in polynomial
    time.

29
NP-Completeness and Reducibility
  • NP-complete problems
  • informally they are the hardest problems in the
    class NP
  • no polynomial time algorithm is known to solve
    any of them
  • but every NP-complete problem is reducible to
    every other NP-complete problem in polynomial time

30
Reducibility
Polynomial time
Set of instances of Problem A
Set of instances of Problem B
Decision problem A is polynomial-time reducible
to decision problem B if a polynomial time
algorithm can be developed which changes each
instance of problem A to an instance of problem B
such that if the answer for an instance of B is
yes, the answer to the corresponding instance of
A is yes.
31
Algorithm from Reduction for Solving Problem A
YES
Polynomial-time reduction algorithm
Polynomial-time algorithm to decide B
?
?
NO
32
A First NP-Complete Problem
  • This technique for showing that a problem is in
    the class NP-Complete requires that we have one
    NP-Complete problem to begin with
  • Circuit satisifiability was the first problem to
    be shown to be in NP-Complete.

33
Lemma 34.3
  • If Q1, Q2 are problems such that
  • Q1 ?p Q2, then Q2 ? P implies Q1 ?P.
  • Proof
  • Suppose A1 is a polynomial time algorithm that
    reduces instances of Q1to instances of Q2 and
    that A2 is a polynomial time algorithm for
    problem Q2. Then the concatenation of A1 and A2
    is a polynomial time algorithm for Q1.

34
An Instance of Problem Q1
Reduction (polynomial)
An Instance of Problem Q2
Polynomial algorithm for Q1
Polynomial algorithm for Q2
Solution for any Instance (YES,NO)
35
NP-Complete
  • A problem that satisfies property 2, but not
    necessarily property 1 is NP-hard.

36
Theorem 34.4
  • If any NP-complete problem is polynomial-time
    solvable, then P NP.
  • If any problem in NP is not polynomial-time
    solvable, then all NP-complete problems are not
    polynomial-time solvable.

37
Proof of Theorem 34.4
  • Proof of Statement 1 If any NP-complete problem
    is polynomial-time solvable, then P NP.
  • Suppose that Q ??P and Q ??NPC.
  • Then, for any Q ??NP, we know that Q ?pQ by
    property 2 of NP-completeness. By Lemma 34.3,
    Q ??P.

P
Q
NPC
Q
NP
Q
P/NP
NPC
38
Proof of Theorem 34.4
  • Proof of Statement 2 If any problem in NP is not
    polynomial-time solvable, then all NP-complete
    problems are not polynomial-time solvable.
  • Suppose that there exists an Q ??NP such that Q
    ?P. Let Q ??NPC be any NP-complete problem and
    suppose that Q ? P.
  • By Lemma 34.3, Q ?pQ and so Q ?P. Contradiction

NPC
Q
P
Q
NPC
39
Importance of Theorem
  • The theorem make the focus of the PNP question
    the NP-complete problems
  • Most theoretical computer scientists believe that
    P ??NP.
  • But, if a polynomial time algorithm exists for
    any NPC problem, then P NP.
  • But, no one has been able to do this so far.

40
NP-Completeness Proofs
  • Suppose that we can show that one problem (Q) is
    in the class NP-complete. Then we can show that
    another problem Q is in the class NP-complete by
    showing that
  • Q is in NP
  • Q can be reduced to Q in polynomial time.
  • Describe an algorithm for the reduction
  • Prove that the answer to Q is yes iff the answer
    to Q is yes
  • Prove that the algorithm for the reduction runs
    in polynomial time

41
Circuit Satisifiability
  • Cook proved that the CIRCUIT-SAT problem is
    NP-complete.
  • 3-SAT was also shown to be NPC by Cook.
  • It is quite difficult to show that a particular
    problem in NP can be reduced to every other
    problem in NP.
  • But once one problem is know to be NP-complete,
    the others are easier.
  • Many other problems have since been proven to be
    NP-complete.

42
Sketch of NP-Completeness Proof for CIRCUIT-SAT
  • Very long and difficult in the details.
  • Used to provide the basis for most other proofs
    of NP-Completeness.
  • Problem domain is combinatorial circuits.

43
Combinatorial Circuit Components
44
Circuit-Satisfiability Problem
  • Given a boolena combinatorial circuit composed of
    AND, OR, and NOT gates, is it satisfiable?

45
Demonstrating Circuit Satisfiability is NP
Complete
  • This proof was done from first principles.
  • It did not depend on the existence of any other
    NPC problems.
  • A problem Q is NP-Complete if
  • 1. It is an element of the class NP
  • 2. Q ?pQ for every Q in NP

46
Step 1 Prove CIRCUIT-SAT ? NP
  • Develop an algorithm A that can verify a solution
    in polynomial time.
  • Inputs for A
  • A standard representation of a combinatorial
    circuit
  • An assignment of input values to the circuit
  • Algorithm A determines the output of the
    combinatorial circuit. Output is 0 or 1.
  • This can be done in polynomial time (linear for a
    clever algorithm.)
  • Therefore CIRCUIT-SAT ?NP

47
Step 2 Reduction
  • We need to show that every other problem in NP
    can be reduced to CIRCUIT-SAT.
  • The basic argument is
  • A combinatorial circuit can be used to implement
    a computer
  • Program
  • Memory
  • Program counter
  • Working storage, etc.
  • You can show that every problem in NP can be
    mapped onto operations that can be represented as
    a sequences of states of combinatorial circuits.
  • You show that this can be done in polynomial
    time.
  • It is not simple to show all of this, but in the
    end you can show that any problem in NP can be
    reduced to CIRCUIT-SAT in polynomial time

48
Implications of CIRCUIT-SAT?NPC
  • CIRCUIT-SAT is the seed problem in NPC
  • Once we know that one problem is in NPC, we can
    use it to demonstrate that other problems are in
    NPC using a simpler procedure.

49
CIRCUIT-SAT
SAT
3-SAT
CLIQUE
SUBSET-SUM
VERTEX-COVER
HAM-CYCLE
TSP
50
EXAMPLE
  • Suppose someone has proven that the problem of
    determining if a graph G has a Hamiltonian cycle
    has been shown to be NP-complete.
  • And we want to show that the Traveling Salesman
    Problem (TSP) is NP-complete.
  • Two steps
  • show that TSP is in the class NP
  • show that HAM reduces to TSP in polynomial time

51
Traveling Salesman Problem
  • TSP Optimization Problem
  • A salesman must visit n cities. He must visit
    each city exactly one time and finish with the
    city at which he starts. Model possible routes
    as an undirected graph with weights on the edges.
  • c(I,j) is cost of traveling from city I to j
  • TSP Decision Problem
  • Change it to a decision problem by making the
    problem to find a route of length at least k.

52
Step 1
  • Show that TSP is in the class NP.
  • If we are given a set of n1 vertices that we
    claim constitute a tour of length at most k, then
    we can
  • check to see that each vertex occurs exactly one
    time
  • check to see that beginning and ending vertices
    are the same
  • check to see that edges exist between adjacent
    vertices in the graph
  • see if the sum of the edges is ? k

53
Step 1 continued
  • We can show that each of these steps can be done
    in polynomial time
  • Therefore TSP is an element of NP

54
Step 2
  • Now we need to show that any HAM problem can be
    reduced to a TSP problem.

Reduction
HAM
TSP
Problem we want to show to be NPC
Known NPC Problem
55
The Two Problems
  • HAM
  • A Hamiltonian cycle of an undirected graph
    G(V,E) is a simple cycle that contains each
    vertex in V.
  • The Hamiltonian-cycle problem asks if a given
    graph contains a Hamiltonian cycle.
  • TSP
  • A TSP circuit in of a undirected graph with edge
    weights visits each vertex exactly one time and
    begins and ends at the same vertex.
  • The TSP problem asks if a given graph contains a
    TSP circuit of length less than k.

56
The Reduction from A to B
  • Consider an instance of problem A. Show how that
    instance can be changed to an instance of B.
  • Prove that the answer to B is yes iff the answer
    to Q is yes
  • Prove that the algorithm for the reduction runs
    in polynomial time.

57
The Reduction Creating an Instance
  • Now we need to show that any HAM problem can be
    reduced to a TSP problem.
  • Let G(V,E) be an instance of HAM
  • Construct an instance of TSP as follows
  • Form the complete graph G(V,E)
  • Define the cost of the edges as
  • c(i,j) 0 if (i,j) is in E, 1 otherwise
  • We look for a tour of length 0

58
G
G
TSP
HAM
59
Step 2 continued
  • Now we must show that graph G has a Hamiltonian
    cycle iff G has a tour of cost at most 0.
  • G has Hamiltonian cycle implies G has tour of
    cost at most n.
  • Suppose that graph G has a Hamiltonian cycle h.
  • Each edge in h belongs to E and thus there is an
    edge from vi to vi1 with a cost of 0 in G
  • Thus, h is a tour in G with a cost of 0.

60
Step 2 continued
  • G has a tour of cost at most 0 implies G has as
    Hamiltonian cycle.
  • Suppose that G has a tour h of cost at most 0.
  • Since the cost of the edges is 0 or 1, none of
    the edges can have a cost of 1if the cost of the
    tour is at most 0
  • Therefore all edges are in E.
  • So h visits each vertex exactly once, contains
    only edges in E, and is therefore a Hamiltonian
    cycle in G.
About PowerShow.com