Problem Complexity and NP-Complete Problems - PowerPoint PPT Presentation

Loading...

PPT – Problem Complexity and NP-Complete Problems PowerPoint presentation | free to download - id: 507c44-MWI0M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Problem Complexity and NP-Complete Problems

Description:

Presented by Ming-Tsung Hsu Problem Complexity Overview Complexity theory Part of the theory of computation Dealing with the resources required to solve a given ... – PowerPoint PPT presentation

Number of Views:37
Avg rating:3.0/5.0
Slides: 76
Provided by: oplabImN
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Problem Complexity and NP-Complete Problems


1
Problem Complexity and NP-Complete Problems
  • Presented by Ming-Tsung Hsu

2
I cant find an efficient algorithm, I guess
Im just too dumb
3
I cant find an efficient algorithm, because
no such algorithm is possible
4
I cant find an efficient algorithm, but
neither can all these famous people
5
Problem Complexity
6
Overview
  • Complexity theory
  • Part of the theory of computation
  • Dealing with the resources required to solve a
    given problem
  • time (how many steps it takes to solve a problem)
  • space (how much memory it takes)
  • Computational difficulty of computable functions

7
Relations between Problems, Algorithms, and
Programs
Problem
Algorithm
. . . .
Algorithm
. . . .
Program
Program
Program
Program
. . . .
  • A single "problem" is an entire set of related
    questions, where each question is a finite-length
    string
  • A particular question is called an instance

8
Time Complexity
  • Number of steps that it takes to solve an
    instance of the problem
  • Function of the size of the input
  • Using the most efficient algorithm
  • Exact number of steps will depend on exactly what
    machine or language is being used
  • To avoid that problem, generally use Big O
    notation

9
Time Complexity (contd)
  • Worst-case
  • an upper bound on the running time for any input
  • Average-case
  • we shall assume that all inputs of a given size
    are equally likely
  • Best-case
  • to get the lower bound

10
Time Complexity (contd)
  • Sequential search in a list of size n
  • worst-case n times
  • best-case 1 times
  • average-case

11
Asymptotic Notation - O-notation (Big O, Order
of)
  • Asymptotic upper bound
  • Definition For a given function g(n), denoted by
    O(g(n)) is the set of functions O(g(n))
    there exist positive constants c and n0 such
    that f(n) lt cg(n) for all n gt n0

12
Asymptotic Notation - O-notation
  • Asymptotic lower bound
  • Definition For a given function g(n), denoted by
    O(g(n)) is the set of functions ?(g(n)) there
    exist positive constants c and n0 such that f(n)
    gt cg(n) for all n gt n0

13
Asymptotic Notation ?-notation
  • Definition For a given function g(n), denoted by
    ?(g(n)) is the set of functions ?(g(n))
    there exist positive constants c1, c2, and n0
    such that c1g(n) lt f(n) lt c2g(n) for all n gt
    n0

14
Cost and Complexity
  • Algorithm complexity can be expressed in Order
    notation, e.g. at what rate does work grow with
    N?
  • O(1) Constant
  • O(logN) Sub-linear
  • O(N) Linear
  • O(NlogN) Nearly linear
  • O(N2) Quadratic
  • O(XN) Exponential
  • But, for a given problem, how do we know if a
    better algorithm is possible?

15
Practical Complexities
  • 109 instructions/second computer

16
Impractical Complexities
  • 109 instructions/second computer

17
Faster Computer Vs Better Algorithm
  • Algorithmic improvement more useful
  • than hardware improvement.
  • E.g. 2n to n3

18
The Problem of Sorting
  • For example, in discussing the problem of
    sorting
  • Algorithms
  • Bubble-sort O(N2)
  • Merge-sort O(N Log N)
  • Quick-sort O(N Log N)
  • Can we do better than O(N Log N)? ? What is the
    problem complexity?

19
Algorithm vs. Problem Complexity
  • Algorithm complexity is defined by analysis of an
    algorithm
  • Problem complexity is defined by
  • An upper bound defined by an algorithm (worst
    case)
  • A lower bound defined by a proof (A lot of
    problems are still unknown)

20
The Upper Bound
  • Defined by an algorithm
  • Defines that we know we can do at least this good
  • Perhaps we can do better
  • Lowered by a better algorithm
  • For problem X, the best algorithm was O(N3), but
    my new algorithm is O(N2).

21
The Lower Bound
  • Defined by a proof
  • Defines that we know we can do no better than
    this
  • It may be worse
  • Raised by a better proof
  • For problem X, the strongest proof showed that
    it required O(N), but my new, stronger proof
    shows that it requires at least O(N2).
  • Might get harder and harder

22
Upper and Lower Bounds
  • The Upper bound is the best algorithmic solution
    that has been found for a problem
  • Whats the best that we know we can do?
  • The Lower bound is the best solution that is
    theoretically possible.
  • What cost can we prove is necessary?

23
Changing the Bounds
Lowered by better algorithm
Upper bound
Lower bound
Raised by better proof
24
(No Transcript)
25
Closed Problems
  • The upper and lower bounds are identical

Upper bound
The inherent complexity of problem
Lower bound
26
Closed Problems (contd)
  • Better algorithms are still possible
  • Better algorithms will not provide an improvement
    detectable by Big O
  • Better algorithms can improve the constant costs
    hidden in Big O characterizations

27
Open Problems
  • The upper and lower bounds differ.

Lowered by better algorithm
Upper bound
Unknown
Who Falls Short?
Lower bound
Raised by better proof
28
Open Problems (contd)
  • D. Harel. Algorithmics The Spirit of Computing.
    Addison-Wesley, 2nd edition,1992
  • . . . if a problem gives rise to a. . . gap, the
    deficiency is not in the problem but in our
    knowledge about it. We have failed either in
    finding the best algorithm for it or in proving
    that a better one does not exist, or in both

29
Examples - Closed
  • Problem Searching an unordered list of n items
  • Upper bound O(n) comparisons (from linear
    search)
  • Lower bound O(n) comparisons
  • No gap so we know the problem complexity O(n)
  • Problem Searching an ordered list of n items
  • Upper bound O(log n) comparisons (from binary
    search)
  • Lower bound O(log n) comparisons
  • No gap so we know the problem complexity O(log
    n)

30
Examples - Closed (contd)
  • Problem Sorting n arbitrary elements
  • Upper bound O(n log n) comparisons (from,
    e.g., merge sort)
  • Lower bound O(n log n) comparisons
  • No gap so we know the problem complexity O(n
    log n)
  • Problem Towers of Hanoi for n disks (n gt 1)
  • Upper bound O(2n) moves
  • Lower bound O(2n) moves
  • No gap so we know the problem complexity O(2n)

31
Examples - Open
  • Problem Multiplication of two integers, where n
    is the number of digits
  • Upper bound O(nlognloglogn)
  • Lower bound O(n)
  • Theres a gap (but only a small one)
  • Problem Finding the minimum spanning tree in a
    graph of n edges and m vertices.
  • Upper bound O(nlogn) or O(nmlogm)
  • Lower bound O(n)
  • Theres a gap (but only a small one)
  • Do not be fooled by the last two examples into
    thinking that all gaps are small!

32
Tractable vs. Intractable
  • Problems are tractable if the upper bounds and
    lower bounds have only polynomial factors
  • O (log N)
  • O (N)
  • O (NK) where K is a constant
  • Problems are intractable if the upper bounds and
    lower bounds have an exponential factor (are
    solvable in theory, but can't be solved in
    practice)
  • O (N!)
  • O (NN)
  • O (2N)

33
Terminology
  • Polynomial algorithms are reasonable
  • Polynomial problems are tractable
  • Exponential algorithms are unreasonable
  • Exponential problems are intractable

34
Terminology
35
NP-Complete Problems
36
Definitions of P, NP
  • A decision problem is a problem where the answer
    is always YES/NO
  • An optimization problem is a problem can be many
    possible solutions. Each solution has a value,
    and we wish to fond a solution with the optimal
    (maximum or minimum) value
  • We can re-cast an optimization problem to a
    decision problem by using loop
  • EX Find Max Clique in a graph ? Is there a
    clique which size is kn, n-1, ?

37
Definitions (contd)
  • A deterministic algorithm is an algorithm which,
    in informal terms, behaves predictably. If it
    runs on a particular input, it will always
    produce the same correct output, and the
    underlying machine will always pass through the
    same sequence of states
  • A non-deterministic algorithm has two stages
  • 1. Guessing (in nondeterministic polynomial time
    with correct guessing) Stage
  • 2. Verification (in deterministic polynomial
    time) Stage

38
Definitions (contd)
  • EX A (3, 11, 2, 5, 8, 16, , 200 ), Is the
    x5 in A?
  • deterministic algorithm
  • for i1 to n if A(i) x then print (i)
    and return true next return false
  • non-deterministic algorithm
  • j ? choice (1n) if A(j) x then print (j)
    success endif print (0) failure

39
Certificates
  • Returning true in order to show that the
    solution can be made, we only have to show one
    solution that works
  • This is called a certificate.
  • Returning false in order to show that the
    solution cannot be made, we must test all
    solution.

40
Oracles
  • If we could make the right decision at all
    decision points, then we can determine whether a
    solution is possible very quickly!
  • If the found solution is valid, then True
  • If the found solution is invalid, then False
  • If we could find the certificates quickly, NPC
    problems would become tractable O(N)
  • This (magic) process that can always make the
    right guess is called an Oracle.

41
Determinism vs. Nondeterminism
  • Nondeterministic algorithms produce an answer by
    a series of correct guesses
  • Deterministic algorithms (like those that a
    computer executes) make decisions based on
    information.

42
Definitions (contd)
  • The complexity class P is the set of decision
    problems that can be solved by a deterministic
    machine (algorithm) in polynomial time
  • The complexity class NP is the set of decision
    problems that can be solved by a
    non-deterministic machine (algorithm) in
    polynomial time
  • Since deterministic algorithms are just a special
    case of deterministic ones ? P?NP
  • The big question Does P NP?

43
Definitions (contd)
44
Problems that Cross the Line
  • What if a problem has
  • An exponential upper bound
  • A polynomial lower bound
  • We have only found exponential algorithms, so it
    appears to be intractable.
  • But... we cant prove that an exponential
    solution is needed, we cant prove that a
    polynomial algorithm cannot be developed, so we
    cant say the problem is intractable...

45
Reduction
  • A problem P can be reduced to another problem Q
    if any instance of P can be rephrased to an
    instance of Q, the solution to which provides a
    solution to the instance of P
  • This rephrasing is called a transformation
  • If P is polynomial-time reducible to Q, we denote
    this P ? Q
  • Intuitively If P reduces in polynomial time to
    Q, P is no harder to solve than Q
  • EX Maximum ? Sorting but Sorting cannot be
    reduced to Maximum

46
The Boolean Satisfiability Problem (SAT)
  • An instance of the problem is a Boolean
    expression written using only AND, OR, NOT,
    variables, and parentheses
  • Question given the expression, is there some
    assignment of TRUE and FALSE values to the
    variables will make the entire expression true?
  • Clause OR together a group of literals
  • Conjunctive normal form (CNF) formulas that are
    a conjunction (AND) of clauses
  • SAT ? NP O(2nm) (n of literals, m of
    clauses)

47
SAT (contd)
  • EX CNF a1 V a2 V a3 (clause1) a1
    (clause2) ? (0, 0, 1)
    satisfied a2 (clause3)

48
Cooks Theorem
  • NP P iff SAT ? P
  • pf (?) ? SAT ? NP ? SAT?P (?) Difficult (every
    NP problem can be reduced to SAT problem)

49
NP-Complete and NP-Hard
  • A problem P is NP-Complete if P ? NP and every NP
    problem can be reduced to P
  • P ? NP and X ? P for all X ?NP
  • If any NP-Complete problem can be solved in
    polynomial time, then NP P
  • SAT problem is an NP-complete problem
  • A problem P is NP-Hard if every NP problem can be
    reduced to P
  • We say P is NP-Complete if P is NP-Hard and P ?
    NP

50
NP-Complete
  • NP-Complete comes from
  • Nondeterministic Algorithm in Polynomial time
  • Complete - Solve one, Solve them all
  • There are more NP-Complete problems than provably
    intractable problems.

51
NP-Hard
  • NP-Hard comes from
  • Nondeterministic Algorithm in Polynomial time
  • Hard - as "hard" as any problem in NP

52
NP-Complete and NP-Hard (contd)
NP NPC
NPH
53
Example NP-Complete Problems
  • Path-Finding (Traveling salesman)
  • Map coloring
  • Scheduling and Matching (bin packing)
  • 2-D arrangement problems
  • Planning problems (pert planning)
  • Clique


54
Traveling Salesman
55
5-Clique
56
Map Coloring
57
The Knapsack Problem
  • The 0-1 knapsack problem
  • A thief must choose among n items, where the ith
    item worth vi dollars and weighs wi pounds
  • Carrying at most W pounds, maximize value
  • EX B 13,17,19,21,24,27,36,42, S 93, Find
    X(X1,X2,,X8), where Xi0 or 1, i1,2,3,,8 such
    that SXiBi S
  • A variation, the fractional knapsack problem
  • Thief can take fractions of items
  • Think of items in 0-1 problem as gold ingots, in
    fractional problem as buckets of gold dust

58
Class Scheduling Problem
  • With N teachers with certain hour restrictions M
    classes to be scheduled, can we
  • Schedule all the classes
  • Make sure that no two teachers teach the same
    class at the same time
  • No teacher is scheduled to teach two classes at
    once

59
Proving NP-Completeness
  • Show that the problem A is in NP. (i.e. Show that
    a certificate can be verified in polynomial time)
  • Assume A is not NP complete
  • Show how to convert an existing NPC problem into
    the problem A (in polynomial time)
  • P is NP-Complete
  • P ? A
  • Proving that reduction is polynomial
  • ? A is NP-Complete
  • If we can do it weve done the proof!
  • If we can turn an existing NP-complete problem
    into our problem in polynomial time... ???

60
3-SAT is NPC ?
  • 3-SAT is NP
  • SAT ? 3-SAT (by construct sets of unsatisfiable
    clauses)
  • Clause contains only one literal Li
  • Q1 V Q2 Li V Q1 V Q2
    O1 V Q2 Li V O1 V Q2 Q1 V
    O2 ? Li V Q1 V O2 O1 V O2
    Li V O1 V O2
  • Clause contains two literals Li and Lj
  • Q1 Li V Lj V
    Q1 O1 ? Li V Lj V
    O1

61
3-SAT is NPC? (contd)
  • Clause contains three literals ? do nothing
  • Clause contains more than three literals (L1, L2,
    , Ln )
  • Reduction is O(mn) (m of clauses , n of
    literals)

62
3-SAT is NPC? (contd)
  • Example

63
3-SAT is NPC? (contd)
L1 L2 L3 L4 L5 Y1 Y2 Y3 Y4 Y5
1 1 1 1 0 0 0 0 0 0
1 1 0 0 0 0 0 0 0 1
1 1 0 1 0 0 0 0 0 0
1 1 0 0 1 0 0 0 0 0
1 1 0 1 1 0 0 0 0 0
1 1 1 1 1 0 0 0 0 0
1 0 0 0 0 0 0 0 0 1
1 0 0 1 0 0 0 0 0 0
1 0 0 0 1 0 0 0 0 0
1 0 0 1 1 0 0 0 0 0
L1 L2 L3 L4 L5
1 1 1 1 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1 1 0 1 1
1 1 1 1 1
1 0 0 0 0
1 0 0 1 0
1 0 0 0 1
1 0 0 1 1
64
Clique Decision Problem (CDP) is NPC
  • Construct a graph G (V,E) from
    such that G will have a clique of size
    at least m iff F is satisfiable (SAT ? CDP)
  • Construction Rule
  • A literal in a clause is a vertex (node) in G
  • A edge is a intersection of two clauses
  • An assignment of F
  • (a) One literal is true then the clause is true
  • (b) A A does not exist
  • ? Add edges between any two nodes exclusive of
  • nodes are in the same clause and nodes are
    exclusive

65
CDP (contd)
  • Example F (a1 V a2 V a3) (a1 V a2 V a3)
    (a2) and G constructed by F
  • F is satisfied by (1,0,0) or (0,0,1) ? G has a
    clique of size at least 3
  • m 3 ? F is satisfied

a1, 1
a1, 2
a2, 1
a2, 2
a3, 1
a3, 2
a2, 3
66
CDP (contd)
  • Proof support F has m clauses
  • (? ) F is satisfiable ? exists an assignment
  • the literals in this assignment forms a clique
    with size equal m (a complete subgraph of G with
    degree m-1)
  • (?)If we can find a clique C with size equal m
  • Every clause has exactly one literal in C
  • Any two vertices in C are jointed and each edge
    is the intersection of two clauses ? Every
    clause is true by literals in C ? F is satisfied

67
Become Famous!
  • To get famous in a hurry, for any NP-Complete
    problem
  • Raise the lower bound (via a stronger proof)
  • Lower the upper bound (via a better algorithm)
  • Theyll be naming buildings after you before you
    are dead!

68
Decidable vs. Undecidable Problems
69
Decidable Problems
  • We now have three categories
  • Tractable problems
  • NP-Complete problems
  • Intractable problems
  • All of the above have algorithmic solutions, even
    if impractical.

70
Undecidable Problems
  • No algorithmic solution exists
  • Regardless of cost
  • These problems arent computable
  • No answer can be obtained in finite amount of time

71
The Unbounded Tiling Problem
  • What if we took the tiling puzzle and removed the
    bounds
  • For a given number (T) of different kinds of
    tiles
  • Can we arrive at an arrangement that will fill
    any size area?
  • By removing the bounds, this problem becomes
    undecidable.

72
The Halting Problem
  • Given an algorithm A and an input I, will the
    algorithm reach a stopping place?
  • loop
  • exitif (x 1)
  • if (even(x)) then
  • x lt- x div 2
  • else
  • x lt- 3 x 1
  • endloop
  • In general, we cannot solve this problem in
    finite time.

73
Partially and Highly Undecidable
  • Most undecidable problems have finite
    certificates. These are partially decidable.
  • Some undecidable problems do not have finite
    certificates even with an oracle. These are
    called highly undecidable.

74
A Hierarchy of Problems
  • For a given problem, is there or will there ever
    be an algorithmic solution?
  • Problem In Theory In Application
  • Highly Undecidable NO NO
  • Partially Undecidable NO NO
  • Intractable YES NO
  • Tractable YES YES

75
Thanks
About PowerShow.com