Loading...

PPT – Problem Complexity and NP-Complete Problems PowerPoint presentation | free to download - id: 507c44-MWI0M

The Adobe Flash plugin is needed to view this content

Problem Complexity and NP-Complete Problems

- Presented by Ming-Tsung Hsu

I cant find an efficient algorithm, I guess

Im just too dumb

I cant find an efficient algorithm, because

no such algorithm is possible

I cant find an efficient algorithm, but

neither can all these famous people

Problem Complexity

Overview

- Complexity theory
- Part of the theory of computation
- Dealing with the resources required to solve a

given problem - time (how many steps it takes to solve a problem)

- space (how much memory it takes)
- Computational difficulty of computable functions

Relations between Problems, Algorithms, and

Programs

Problem

Algorithm

. . . .

Algorithm

. . . .

Program

Program

Program

Program

. . . .

- A single "problem" is an entire set of related

questions, where each question is a finite-length

string - A particular question is called an instance

Time Complexity

- Number of steps that it takes to solve an

instance of the problem - Function of the size of the input
- Using the most efficient algorithm
- Exact number of steps will depend on exactly what

machine or language is being used - To avoid that problem, generally use Big O

notation

Time Complexity (contd)

- Worst-case
- an upper bound on the running time for any input
- Average-case
- we shall assume that all inputs of a given size

are equally likely - Best-case
- to get the lower bound

Time Complexity (contd)

- Sequential search in a list of size n
- worst-case n times
- best-case 1 times
- average-case

Asymptotic Notation - O-notation (Big O, Order

of)

- Asymptotic upper bound
- Definition For a given function g(n), denoted by

O(g(n)) is the set of functions O(g(n))

there exist positive constants c and n0 such

that f(n) lt cg(n) for all n gt n0

Asymptotic Notation - O-notation

- Asymptotic lower bound
- Definition For a given function g(n), denoted by

O(g(n)) is the set of functions ?(g(n)) there

exist positive constants c and n0 such that f(n)

gt cg(n) for all n gt n0

Asymptotic Notation ?-notation

- Definition For a given function g(n), denoted by

?(g(n)) is the set of functions ?(g(n))

there exist positive constants c1, c2, and n0

such that c1g(n) lt f(n) lt c2g(n) for all n gt

n0

Cost and Complexity

- Algorithm complexity can be expressed in Order

notation, e.g. at what rate does work grow with

N? - O(1) Constant
- O(logN) Sub-linear
- O(N) Linear
- O(NlogN) Nearly linear
- O(N2) Quadratic
- O(XN) Exponential
- But, for a given problem, how do we know if a

better algorithm is possible?

Practical Complexities

- 109 instructions/second computer

Impractical Complexities

- 109 instructions/second computer

Faster Computer Vs Better Algorithm

- Algorithmic improvement more useful
- than hardware improvement.
- E.g. 2n to n3

The Problem of Sorting

- For example, in discussing the problem of

sorting - Algorithms
- Bubble-sort O(N2)
- Merge-sort O(N Log N)
- Quick-sort O(N Log N)
- Can we do better than O(N Log N)? ? What is the

problem complexity?

Algorithm vs. Problem Complexity

- Algorithm complexity is defined by analysis of an

algorithm - Problem complexity is defined by
- An upper bound defined by an algorithm (worst

case) - A lower bound defined by a proof (A lot of

problems are still unknown)

The Upper Bound

- Defined by an algorithm
- Defines that we know we can do at least this good
- Perhaps we can do better
- Lowered by a better algorithm
- For problem X, the best algorithm was O(N3), but

my new algorithm is O(N2).

The Lower Bound

- Defined by a proof
- Defines that we know we can do no better than

this - It may be worse
- Raised by a better proof
- For problem X, the strongest proof showed that

it required O(N), but my new, stronger proof

shows that it requires at least O(N2). - Might get harder and harder

Upper and Lower Bounds

- The Upper bound is the best algorithmic solution

that has been found for a problem - Whats the best that we know we can do?
- The Lower bound is the best solution that is

theoretically possible. - What cost can we prove is necessary?

Changing the Bounds

Lowered by better algorithm

Upper bound

Lower bound

Raised by better proof

(No Transcript)

Closed Problems

- The upper and lower bounds are identical

Upper bound

The inherent complexity of problem

Lower bound

Closed Problems (contd)

- Better algorithms are still possible
- Better algorithms will not provide an improvement

detectable by Big O - Better algorithms can improve the constant costs

hidden in Big O characterizations

Open Problems

- The upper and lower bounds differ.

Lowered by better algorithm

Upper bound

Unknown

Who Falls Short?

Lower bound

Raised by better proof

Open Problems (contd)

- D. Harel. Algorithmics The Spirit of Computing.

Addison-Wesley, 2nd edition,1992 - . . . if a problem gives rise to a. . . gap, the

deficiency is not in the problem but in our

knowledge about it. We have failed either in

finding the best algorithm for it or in proving

that a better one does not exist, or in both

Examples - Closed

- Problem Searching an unordered list of n items
- Upper bound O(n) comparisons (from linear

search) - Lower bound O(n) comparisons
- No gap so we know the problem complexity O(n)
- Problem Searching an ordered list of n items
- Upper bound O(log n) comparisons (from binary

search) - Lower bound O(log n) comparisons
- No gap so we know the problem complexity O(log

n)

Examples - Closed (contd)

- Problem Sorting n arbitrary elements
- Upper bound O(n log n) comparisons (from,

e.g., merge sort) - Lower bound O(n log n) comparisons
- No gap so we know the problem complexity O(n

log n) - Problem Towers of Hanoi for n disks (n gt 1)
- Upper bound O(2n) moves
- Lower bound O(2n) moves
- No gap so we know the problem complexity O(2n)

Examples - Open

- Problem Multiplication of two integers, where n

is the number of digits - Upper bound O(nlognloglogn)
- Lower bound O(n)
- Theres a gap (but only a small one)
- Problem Finding the minimum spanning tree in a

graph of n edges and m vertices. - Upper bound O(nlogn) or O(nmlogm)
- Lower bound O(n)
- Theres a gap (but only a small one)
- Do not be fooled by the last two examples into

thinking that all gaps are small!

Tractable vs. Intractable

- Problems are tractable if the upper bounds and

lower bounds have only polynomial factors - O (log N)
- O (N)
- O (NK) where K is a constant
- Problems are intractable if the upper bounds and

lower bounds have an exponential factor (are

solvable in theory, but can't be solved in

practice) - O (N!)
- O (NN)
- O (2N)

Terminology

- Polynomial algorithms are reasonable
- Polynomial problems are tractable
- Exponential algorithms are unreasonable
- Exponential problems are intractable

Terminology

NP-Complete Problems

Definitions of P, NP

- A decision problem is a problem where the answer

is always YES/NO - An optimization problem is a problem can be many

possible solutions. Each solution has a value,

and we wish to fond a solution with the optimal

(maximum or minimum) value - We can re-cast an optimization problem to a

decision problem by using loop - EX Find Max Clique in a graph ? Is there a

clique which size is kn, n-1, ?

Definitions (contd)

- A deterministic algorithm is an algorithm which,

in informal terms, behaves predictably. If it

runs on a particular input, it will always

produce the same correct output, and the

underlying machine will always pass through the

same sequence of states - A non-deterministic algorithm has two stages
- 1. Guessing (in nondeterministic polynomial time

with correct guessing) Stage - 2. Verification (in deterministic polynomial

time) Stage

Definitions (contd)

- EX A (3, 11, 2, 5, 8, 16, , 200 ), Is the

x5 in A? - deterministic algorithm
- for i1 to n if A(i) x then print (i)

and return true next return false - non-deterministic algorithm
- j ? choice (1n) if A(j) x then print (j)

success endif print (0) failure

Certificates

- Returning true in order to show that the

solution can be made, we only have to show one

solution that works - This is called a certificate.
- Returning false in order to show that the

solution cannot be made, we must test all

solution.

Oracles

- If we could make the right decision at all

decision points, then we can determine whether a

solution is possible very quickly! - If the found solution is valid, then True
- If the found solution is invalid, then False
- If we could find the certificates quickly, NPC

problems would become tractable O(N) - This (magic) process that can always make the

right guess is called an Oracle.

Determinism vs. Nondeterminism

- Nondeterministic algorithms produce an answer by

a series of correct guesses - Deterministic algorithms (like those that a

computer executes) make decisions based on

information.

Definitions (contd)

- The complexity class P is the set of decision

problems that can be solved by a deterministic

machine (algorithm) in polynomial time - The complexity class NP is the set of decision

problems that can be solved by a

non-deterministic machine (algorithm) in

polynomial time - Since deterministic algorithms are just a special

case of deterministic ones ? P?NP - The big question Does P NP?

Definitions (contd)

Problems that Cross the Line

- What if a problem has
- An exponential upper bound
- A polynomial lower bound
- We have only found exponential algorithms, so it

appears to be intractable. - But... we cant prove that an exponential

solution is needed, we cant prove that a

polynomial algorithm cannot be developed, so we

cant say the problem is intractable...

Reduction

- A problem P can be reduced to another problem Q

if any instance of P can be rephrased to an

instance of Q, the solution to which provides a

solution to the instance of P - This rephrasing is called a transformation
- If P is polynomial-time reducible to Q, we denote

this P ? Q - Intuitively If P reduces in polynomial time to

Q, P is no harder to solve than Q - EX Maximum ? Sorting but Sorting cannot be

reduced to Maximum

The Boolean Satisfiability Problem (SAT)

- An instance of the problem is a Boolean

expression written using only AND, OR, NOT,

variables, and parentheses - Question given the expression, is there some

assignment of TRUE and FALSE values to the

variables will make the entire expression true? - Clause OR together a group of literals
- Conjunctive normal form (CNF) formulas that are

a conjunction (AND) of clauses - SAT ? NP O(2nm) (n of literals, m of

clauses)

SAT (contd)

- EX CNF a1 V a2 V a3 (clause1) a1

(clause2) ? (0, 0, 1)

satisfied a2 (clause3)

Cooks Theorem

- NP P iff SAT ? P
- pf (?) ? SAT ? NP ? SAT?P (?) Difficult (every

NP problem can be reduced to SAT problem)

NP-Complete and NP-Hard

- A problem P is NP-Complete if P ? NP and every NP

problem can be reduced to P - P ? NP and X ? P for all X ?NP
- If any NP-Complete problem can be solved in

polynomial time, then NP P - SAT problem is an NP-complete problem
- A problem P is NP-Hard if every NP problem can be

reduced to P - We say P is NP-Complete if P is NP-Hard and P ?

NP

NP-Complete

- NP-Complete comes from
- Nondeterministic Algorithm in Polynomial time
- Complete - Solve one, Solve them all
- There are more NP-Complete problems than provably

intractable problems.

NP-Hard

- NP-Hard comes from
- Nondeterministic Algorithm in Polynomial time
- Hard - as "hard" as any problem in NP

NP-Complete and NP-Hard (contd)

NP NPC

NPH

Example NP-Complete Problems

- Path-Finding (Traveling salesman)
- Map coloring
- Scheduling and Matching (bin packing)
- 2-D arrangement problems
- Planning problems (pert planning)
- Clique

Traveling Salesman

5-Clique

Map Coloring

The Knapsack Problem

- The 0-1 knapsack problem
- A thief must choose among n items, where the ith

item worth vi dollars and weighs wi pounds - Carrying at most W pounds, maximize value
- EX B 13,17,19,21,24,27,36,42, S 93, Find

X(X1,X2,,X8), where Xi0 or 1, i1,2,3,,8 such

that SXiBi S - A variation, the fractional knapsack problem
- Thief can take fractions of items
- Think of items in 0-1 problem as gold ingots, in

fractional problem as buckets of gold dust

Class Scheduling Problem

- With N teachers with certain hour restrictions M

classes to be scheduled, can we - Schedule all the classes
- Make sure that no two teachers teach the same

class at the same time - No teacher is scheduled to teach two classes at

once

Proving NP-Completeness

- Show that the problem A is in NP. (i.e. Show that

a certificate can be verified in polynomial time) - Assume A is not NP complete
- Show how to convert an existing NPC problem into

the problem A (in polynomial time) - P is NP-Complete
- P ? A
- Proving that reduction is polynomial
- ? A is NP-Complete
- If we can do it weve done the proof!
- If we can turn an existing NP-complete problem

into our problem in polynomial time... ???

3-SAT is NPC ?

- 3-SAT is NP
- SAT ? 3-SAT (by construct sets of unsatisfiable

clauses) - Clause contains only one literal Li
- Q1 V Q2 Li V Q1 V Q2

O1 V Q2 Li V O1 V Q2 Q1 V

O2 ? Li V Q1 V O2 O1 V O2

Li V O1 V O2 - Clause contains two literals Li and Lj
- Q1 Li V Lj V

Q1 O1 ? Li V Lj V

O1

3-SAT is NPC? (contd)

- Clause contains three literals ? do nothing
- Clause contains more than three literals (L1, L2,

, Ln ) - Reduction is O(mn) (m of clauses , n of

literals)

3-SAT is NPC? (contd)

- Example

3-SAT is NPC? (contd)

L1 L2 L3 L4 L5 Y1 Y2 Y3 Y4 Y5

1 1 1 1 0 0 0 0 0 0

1 1 0 0 0 0 0 0 0 1

1 1 0 1 0 0 0 0 0 0

1 1 0 0 1 0 0 0 0 0

1 1 0 1 1 0 0 0 0 0

1 1 1 1 1 0 0 0 0 0

1 0 0 0 0 0 0 0 0 1

1 0 0 1 0 0 0 0 0 0

1 0 0 0 1 0 0 0 0 0

1 0 0 1 1 0 0 0 0 0

L1 L2 L3 L4 L5

1 1 1 1 0

1 1 0 0 0

1 1 0 1 0

1 1 0 0 1

1 1 0 1 1

1 1 1 1 1

1 0 0 0 0

1 0 0 1 0

1 0 0 0 1

1 0 0 1 1

Clique Decision Problem (CDP) is NPC

- Construct a graph G (V,E) from

such that G will have a clique of size

at least m iff F is satisfiable (SAT ? CDP) - Construction Rule
- A literal in a clause is a vertex (node) in G
- A edge is a intersection of two clauses
- An assignment of F
- (a) One literal is true then the clause is true

- (b) A A does not exist
- ? Add edges between any two nodes exclusive of
- nodes are in the same clause and nodes are

exclusive

CDP (contd)

- Example F (a1 V a2 V a3) (a1 V a2 V a3)

(a2) and G constructed by F - F is satisfied by (1,0,0) or (0,0,1) ? G has a

clique of size at least 3 - m 3 ? F is satisfied

a1, 1

a1, 2

a2, 1

a2, 2

a3, 1

a3, 2

a2, 3

CDP (contd)

- Proof support F has m clauses
- (? ) F is satisfiable ? exists an assignment
- the literals in this assignment forms a clique

with size equal m (a complete subgraph of G with

degree m-1) - (?)If we can find a clique C with size equal m
- Every clause has exactly one literal in C
- Any two vertices in C are jointed and each edge

is the intersection of two clauses ? Every

clause is true by literals in C ? F is satisfied

Become Famous!

- To get famous in a hurry, for any NP-Complete

problem - Raise the lower bound (via a stronger proof)
- Lower the upper bound (via a better algorithm)
- Theyll be naming buildings after you before you

are dead!

Decidable vs. Undecidable Problems

Decidable Problems

- We now have three categories
- Tractable problems
- NP-Complete problems
- Intractable problems
- All of the above have algorithmic solutions, even

if impractical.

Undecidable Problems

- No algorithmic solution exists
- Regardless of cost
- These problems arent computable
- No answer can be obtained in finite amount of time

The Unbounded Tiling Problem

- What if we took the tiling puzzle and removed the

bounds - For a given number (T) of different kinds of

tiles - Can we arrive at an arrangement that will fill

any size area? - By removing the bounds, this problem becomes

undecidable.

The Halting Problem

- Given an algorithm A and an input I, will the

algorithm reach a stopping place? - loop
- exitif (x 1)
- if (even(x)) then
- x lt- x div 2
- else
- x lt- 3 x 1
- endloop
- In general, we cannot solve this problem in

finite time.

Partially and Highly Undecidable

- Most undecidable problems have finite

certificates. These are partially decidable. - Some undecidable problems do not have finite

certificates even with an oracle. These are

called highly undecidable.

A Hierarchy of Problems

- For a given problem, is there or will there ever

be an algorithmic solution? - Problem In Theory In Application
- Highly Undecidable NO NO
- Partially Undecidable NO NO
- Intractable YES NO
- Tractable YES YES

Thanks