# Problem Complexity and NP-Complete Problems - PowerPoint PPT Presentation

PPT – Problem Complexity and NP-Complete Problems PowerPoint presentation | free to download - id: 507c44-MWI0M

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Problem Complexity and NP-Complete Problems

Description:

### Presented by Ming-Tsung Hsu Problem Complexity Overview Complexity theory Part of the theory of computation Dealing with the resources required to solve a given ... – PowerPoint PPT presentation

Number of Views:37
Avg rating:3.0/5.0
Slides: 76
Provided by: oplabImN
Category:
Tags:
Transcript and Presenter's Notes

Title: Problem Complexity and NP-Complete Problems

1
Problem Complexity and NP-Complete Problems
• Presented by Ming-Tsung Hsu

2
I cant find an efficient algorithm, I guess
Im just too dumb
3
I cant find an efficient algorithm, because
no such algorithm is possible
4
I cant find an efficient algorithm, but
neither can all these famous people
5
Problem Complexity
6
Overview
• Complexity theory
• Part of the theory of computation
• Dealing with the resources required to solve a
given problem
• time (how many steps it takes to solve a problem)
• space (how much memory it takes)
• Computational difficulty of computable functions

7
Relations between Problems, Algorithms, and
Programs
Problem
Algorithm
. . . .
Algorithm
. . . .
Program
Program
Program
Program
. . . .
• A single "problem" is an entire set of related
questions, where each question is a finite-length
string
• A particular question is called an instance

8
Time Complexity
• Number of steps that it takes to solve an
instance of the problem
• Function of the size of the input
• Using the most efficient algorithm
• Exact number of steps will depend on exactly what
machine or language is being used
• To avoid that problem, generally use Big O
notation

9
Time Complexity (contd)
• Worst-case
• an upper bound on the running time for any input
• Average-case
• we shall assume that all inputs of a given size
are equally likely
• Best-case
• to get the lower bound

10
Time Complexity (contd)
• Sequential search in a list of size n
• worst-case n times
• best-case 1 times
• average-case

11
Asymptotic Notation - O-notation (Big O, Order
of)
• Asymptotic upper bound
• Definition For a given function g(n), denoted by
O(g(n)) is the set of functions O(g(n))
there exist positive constants c and n0 such
that f(n) lt cg(n) for all n gt n0

12
Asymptotic Notation - O-notation
• Asymptotic lower bound
• Definition For a given function g(n), denoted by
O(g(n)) is the set of functions ?(g(n)) there
exist positive constants c and n0 such that f(n)
gt cg(n) for all n gt n0

13
Asymptotic Notation ?-notation
• Definition For a given function g(n), denoted by
?(g(n)) is the set of functions ?(g(n))
there exist positive constants c1, c2, and n0
such that c1g(n) lt f(n) lt c2g(n) for all n gt
n0

14
Cost and Complexity
• Algorithm complexity can be expressed in Order
notation, e.g. at what rate does work grow with
N?
• O(1) Constant
• O(logN) Sub-linear
• O(N) Linear
• O(NlogN) Nearly linear
• O(XN) Exponential
• But, for a given problem, how do we know if a
better algorithm is possible?

15
Practical Complexities
• 109 instructions/second computer

16
Impractical Complexities
• 109 instructions/second computer

17
Faster Computer Vs Better Algorithm
• Algorithmic improvement more useful
• than hardware improvement.
• E.g. 2n to n3

18
The Problem of Sorting
• For example, in discussing the problem of
sorting
• Algorithms
• Bubble-sort O(N2)
• Merge-sort O(N Log N)
• Quick-sort O(N Log N)
• Can we do better than O(N Log N)? ? What is the
problem complexity?

19
Algorithm vs. Problem Complexity
• Algorithm complexity is defined by analysis of an
algorithm
• Problem complexity is defined by
• An upper bound defined by an algorithm (worst
case)
• A lower bound defined by a proof (A lot of
problems are still unknown)

20
The Upper Bound
• Defined by an algorithm
• Defines that we know we can do at least this good
• Perhaps we can do better
• Lowered by a better algorithm
• For problem X, the best algorithm was O(N3), but
my new algorithm is O(N2).

21
The Lower Bound
• Defined by a proof
• Defines that we know we can do no better than
this
• It may be worse
• Raised by a better proof
• For problem X, the strongest proof showed that
it required O(N), but my new, stronger proof
shows that it requires at least O(N2).
• Might get harder and harder

22
Upper and Lower Bounds
• The Upper bound is the best algorithmic solution
that has been found for a problem
• Whats the best that we know we can do?
• The Lower bound is the best solution that is
theoretically possible.
• What cost can we prove is necessary?

23
Changing the Bounds
Lowered by better algorithm
Upper bound
Lower bound
Raised by better proof
24
(No Transcript)
25
Closed Problems
• The upper and lower bounds are identical

Upper bound
The inherent complexity of problem
Lower bound
26
Closed Problems (contd)
• Better algorithms are still possible
• Better algorithms will not provide an improvement
detectable by Big O
• Better algorithms can improve the constant costs
hidden in Big O characterizations

27
Open Problems
• The upper and lower bounds differ.

Lowered by better algorithm
Upper bound
Unknown
Who Falls Short?
Lower bound
Raised by better proof
28
Open Problems (contd)
• D. Harel. Algorithmics The Spirit of Computing.
• . . . if a problem gives rise to a. . . gap, the
deficiency is not in the problem but in our
knowledge about it. We have failed either in
finding the best algorithm for it or in proving
that a better one does not exist, or in both

29
Examples - Closed
• Problem Searching an unordered list of n items
• Upper bound O(n) comparisons (from linear
search)
• Lower bound O(n) comparisons
• No gap so we know the problem complexity O(n)
• Problem Searching an ordered list of n items
• Upper bound O(log n) comparisons (from binary
search)
• Lower bound O(log n) comparisons
• No gap so we know the problem complexity O(log
n)

30
Examples - Closed (contd)
• Problem Sorting n arbitrary elements
• Upper bound O(n log n) comparisons (from,
e.g., merge sort)
• Lower bound O(n log n) comparisons
• No gap so we know the problem complexity O(n
log n)
• Problem Towers of Hanoi for n disks (n gt 1)
• Upper bound O(2n) moves
• Lower bound O(2n) moves
• No gap so we know the problem complexity O(2n)

31
Examples - Open
• Problem Multiplication of two integers, where n
is the number of digits
• Upper bound O(nlognloglogn)
• Lower bound O(n)
• Theres a gap (but only a small one)
• Problem Finding the minimum spanning tree in a
graph of n edges and m vertices.
• Upper bound O(nlogn) or O(nmlogm)
• Lower bound O(n)
• Theres a gap (but only a small one)
• Do not be fooled by the last two examples into
thinking that all gaps are small!

32
Tractable vs. Intractable
• Problems are tractable if the upper bounds and
lower bounds have only polynomial factors
• O (log N)
• O (N)
• O (NK) where K is a constant
• Problems are intractable if the upper bounds and
lower bounds have an exponential factor (are
solvable in theory, but can't be solved in
practice)
• O (N!)
• O (NN)
• O (2N)

33
Terminology
• Polynomial algorithms are reasonable
• Polynomial problems are tractable
• Exponential algorithms are unreasonable
• Exponential problems are intractable

34
Terminology
35
NP-Complete Problems
36
Definitions of P, NP
• A decision problem is a problem where the answer
is always YES/NO
• An optimization problem is a problem can be many
possible solutions. Each solution has a value,
and we wish to fond a solution with the optimal
(maximum or minimum) value
• We can re-cast an optimization problem to a
decision problem by using loop
• EX Find Max Clique in a graph ? Is there a
clique which size is kn, n-1, ?

37
Definitions (contd)
• A deterministic algorithm is an algorithm which,
in informal terms, behaves predictably. If it
runs on a particular input, it will always
produce the same correct output, and the
underlying machine will always pass through the
same sequence of states
• A non-deterministic algorithm has two stages
• 1. Guessing (in nondeterministic polynomial time
with correct guessing) Stage
• 2. Verification (in deterministic polynomial
time) Stage

38
Definitions (contd)
• EX A (3, 11, 2, 5, 8, 16, , 200 ), Is the
x5 in A?
• deterministic algorithm
• for i1 to n if A(i) x then print (i)
and return true next return false
• non-deterministic algorithm
• j ? choice (1n) if A(j) x then print (j)
success endif print (0) failure

39
Certificates
• Returning true in order to show that the
solution can be made, we only have to show one
solution that works
• This is called a certificate.
• Returning false in order to show that the
solution cannot be made, we must test all
solution.

40
Oracles
• If we could make the right decision at all
decision points, then we can determine whether a
solution is possible very quickly!
• If the found solution is valid, then True
• If the found solution is invalid, then False
• If we could find the certificates quickly, NPC
problems would become tractable O(N)
• This (magic) process that can always make the
right guess is called an Oracle.

41
Determinism vs. Nondeterminism
• Nondeterministic algorithms produce an answer by
a series of correct guesses
• Deterministic algorithms (like those that a
computer executes) make decisions based on
information.

42
Definitions (contd)
• The complexity class P is the set of decision
problems that can be solved by a deterministic
machine (algorithm) in polynomial time
• The complexity class NP is the set of decision
problems that can be solved by a
non-deterministic machine (algorithm) in
polynomial time
• Since deterministic algorithms are just a special
case of deterministic ones ? P?NP
• The big question Does P NP?

43
Definitions (contd)
44
Problems that Cross the Line
• What if a problem has
• An exponential upper bound
• A polynomial lower bound
• We have only found exponential algorithms, so it
appears to be intractable.
• But... we cant prove that an exponential
solution is needed, we cant prove that a
polynomial algorithm cannot be developed, so we
cant say the problem is intractable...

45
Reduction
• A problem P can be reduced to another problem Q
if any instance of P can be rephrased to an
instance of Q, the solution to which provides a
solution to the instance of P
• This rephrasing is called a transformation
• If P is polynomial-time reducible to Q, we denote
this P ? Q
• Intuitively If P reduces in polynomial time to
Q, P is no harder to solve than Q
• EX Maximum ? Sorting but Sorting cannot be
reduced to Maximum

46
The Boolean Satisfiability Problem (SAT)
• An instance of the problem is a Boolean
expression written using only AND, OR, NOT,
variables, and parentheses
• Question given the expression, is there some
assignment of TRUE and FALSE values to the
variables will make the entire expression true?
• Clause OR together a group of literals
• Conjunctive normal form (CNF) formulas that are
a conjunction (AND) of clauses
• SAT ? NP O(2nm) (n of literals, m of
clauses)

47
SAT (contd)
• EX CNF a1 V a2 V a3 (clause1) a1
(clause2) ? (0, 0, 1)
satisfied a2 (clause3)

48
Cooks Theorem
• NP P iff SAT ? P
• pf (?) ? SAT ? NP ? SAT?P (?) Difficult (every
NP problem can be reduced to SAT problem)

49
NP-Complete and NP-Hard
• A problem P is NP-Complete if P ? NP and every NP
problem can be reduced to P
• P ? NP and X ? P for all X ?NP
• If any NP-Complete problem can be solved in
polynomial time, then NP P
• SAT problem is an NP-complete problem
• A problem P is NP-Hard if every NP problem can be
reduced to P
• We say P is NP-Complete if P is NP-Hard and P ?
NP

50
NP-Complete
• NP-Complete comes from
• Nondeterministic Algorithm in Polynomial time
• Complete - Solve one, Solve them all
• There are more NP-Complete problems than provably
intractable problems.

51
NP-Hard
• NP-Hard comes from
• Nondeterministic Algorithm in Polynomial time
• Hard - as "hard" as any problem in NP

52
NP-Complete and NP-Hard (contd)
NP NPC
NPH
53
Example NP-Complete Problems
• Path-Finding (Traveling salesman)
• Map coloring
• Scheduling and Matching (bin packing)
• 2-D arrangement problems
• Planning problems (pert planning)
• Clique

54
Traveling Salesman
55
5-Clique
56
Map Coloring
57
The Knapsack Problem
• The 0-1 knapsack problem
• A thief must choose among n items, where the ith
item worth vi dollars and weighs wi pounds
• Carrying at most W pounds, maximize value
• EX B 13,17,19,21,24,27,36,42, S 93, Find
X(X1,X2,,X8), where Xi0 or 1, i1,2,3,,8 such
that SXiBi S
• A variation, the fractional knapsack problem
• Thief can take fractions of items
• Think of items in 0-1 problem as gold ingots, in
fractional problem as buckets of gold dust

58
Class Scheduling Problem
• With N teachers with certain hour restrictions M
classes to be scheduled, can we
• Schedule all the classes
• Make sure that no two teachers teach the same
class at the same time
• No teacher is scheduled to teach two classes at
once

59
Proving NP-Completeness
• Show that the problem A is in NP. (i.e. Show that
a certificate can be verified in polynomial time)
• Assume A is not NP complete
• Show how to convert an existing NPC problem into
the problem A (in polynomial time)
• P is NP-Complete
• P ? A
• Proving that reduction is polynomial
• ? A is NP-Complete
• If we can do it weve done the proof!
• If we can turn an existing NP-complete problem
into our problem in polynomial time... ???

60
3-SAT is NPC ?
• 3-SAT is NP
• SAT ? 3-SAT (by construct sets of unsatisfiable
clauses)
• Clause contains only one literal Li
• Q1 V Q2 Li V Q1 V Q2
O1 V Q2 Li V O1 V Q2 Q1 V
O2 ? Li V Q1 V O2 O1 V O2
Li V O1 V O2
• Clause contains two literals Li and Lj
• Q1 Li V Lj V
Q1 O1 ? Li V Lj V
O1

61
3-SAT is NPC? (contd)
• Clause contains three literals ? do nothing
• Clause contains more than three literals (L1, L2,
, Ln )
• Reduction is O(mn) (m of clauses , n of
literals)

62
3-SAT is NPC? (contd)
• Example

63
3-SAT is NPC? (contd)
L1 L2 L3 L4 L5 Y1 Y2 Y3 Y4 Y5
1 1 1 1 0 0 0 0 0 0
1 1 0 0 0 0 0 0 0 1
1 1 0 1 0 0 0 0 0 0
1 1 0 0 1 0 0 0 0 0
1 1 0 1 1 0 0 0 0 0
1 1 1 1 1 0 0 0 0 0
1 0 0 0 0 0 0 0 0 1
1 0 0 1 0 0 0 0 0 0
1 0 0 0 1 0 0 0 0 0
1 0 0 1 1 0 0 0 0 0
L1 L2 L3 L4 L5
1 1 1 1 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1 1 0 1 1
1 1 1 1 1
1 0 0 0 0
1 0 0 1 0
1 0 0 0 1
1 0 0 1 1
64
Clique Decision Problem (CDP) is NPC
• Construct a graph G (V,E) from
such that G will have a clique of size
at least m iff F is satisfiable (SAT ? CDP)
• Construction Rule
• A literal in a clause is a vertex (node) in G
• A edge is a intersection of two clauses
• An assignment of F
• (a) One literal is true then the clause is true
• (b) A A does not exist
• ? Add edges between any two nodes exclusive of
• nodes are in the same clause and nodes are
exclusive

65
CDP (contd)
• Example F (a1 V a2 V a3) (a1 V a2 V a3)
(a2) and G constructed by F
• F is satisfied by (1,0,0) or (0,0,1) ? G has a
clique of size at least 3
• m 3 ? F is satisfied

a1, 1
a1, 2
a2, 1
a2, 2
a3, 1
a3, 2
a2, 3
66
CDP (contd)
• Proof support F has m clauses
• (? ) F is satisfiable ? exists an assignment
• the literals in this assignment forms a clique
with size equal m (a complete subgraph of G with
degree m-1)
• (?)If we can find a clique C with size equal m
• Every clause has exactly one literal in C
• Any two vertices in C are jointed and each edge
is the intersection of two clauses ? Every
clause is true by literals in C ? F is satisfied

67
Become Famous!
• To get famous in a hurry, for any NP-Complete
problem
• Raise the lower bound (via a stronger proof)
• Lower the upper bound (via a better algorithm)
• Theyll be naming buildings after you before you

68
Decidable vs. Undecidable Problems
69
Decidable Problems
• We now have three categories
• Tractable problems
• NP-Complete problems
• Intractable problems
• All of the above have algorithmic solutions, even
if impractical.

70
Undecidable Problems
• No algorithmic solution exists
• Regardless of cost
• These problems arent computable
• No answer can be obtained in finite amount of time

71
The Unbounded Tiling Problem
• What if we took the tiling puzzle and removed the
bounds
• For a given number (T) of different kinds of
tiles
• Can we arrive at an arrangement that will fill
any size area?
• By removing the bounds, this problem becomes
undecidable.

72
The Halting Problem
• Given an algorithm A and an input I, will the
algorithm reach a stopping place?
• loop
• exitif (x 1)
• if (even(x)) then
• x lt- x div 2
• else
• x lt- 3 x 1
• endloop
• In general, we cannot solve this problem in
finite time.

73
Partially and Highly Undecidable
• Most undecidable problems have finite
certificates. These are partially decidable.
• Some undecidable problems do not have finite
certificates even with an oracle. These are
called highly undecidable.

74
A Hierarchy of Problems
• For a given problem, is there or will there ever
be an algorithmic solution?
• Problem In Theory In Application
• Highly Undecidable NO NO
• Partially Undecidable NO NO
• Intractable YES NO
• Tractable YES YES

75
Thanks