Problems you shouldn

1
Problems you shouldnt tackle
2
Problem Complexity
3
Relations between Problems, Algorithms, and
Programs
Problem
Algorithm
. . . .
Algorithm
. . . .
Program
Program
Program
Program
. . . .
4
Algorithmic Performance Thus Far

• Some examples thus far
• O(1) Insert to front of linked list
• O(log N) Binary Search
• O(N) Simple/Linear Search
• O(N Log N) Merge/Quick/Heap Sort
• O(N2) Insertion/Bubble Sort
• But it could get worse
• O(N5), O(N2000), etc.

5
An O(N5) Example

• For N 256
• N5 2565 1,100,000,000,000
• If we had a computer that could execute a million
  instructions per second
• 1,100,000 seconds 12.7 days to complete
• But it could get worse

6
The Power of Exponents

• A rich king and a wise peasant

7
The Wise Peasants Pay

• Day(N) Pieces of Grain
• 1 2
• 2 4
• 3 8
• 4 16
• ...

63 9,223,000,000,000,000,000 64
18,450,000,000,000,000,000
8
How Bad is 2N?

• Imagine being able to grow a billion
  (1,000,000,000) pieces of grain a second
• It would take
• 585 years to grow enough grain just for the 64th
  day
• Over a thousand years to fulfill the peasants
  request!

?
9
So the King cut off the peasants head.
10
The Towers of Hanoi
A B
C

• Goal Move stack of rings to another peg
• Rule 1 May move only 1 ring at a time
• Rule 2 May never have larger ring on top of
  smaller ring

11
Towers of Hanoi Solution
Original State
Move 1
Move 2
Move 3
Move 5
Move 4
Move 6
Move 7
12
Towers of Hanoi - Complexity

• For 3 rings we have 7 operations.
• To add a 4th ring, you have to move the upper 3
  rings to one peg, move the 4th ring, then move
  the upper 3 rings on top of it doubling the
  amount of work.
• In general, the cost is 2N 1 O(2N)
• This grows incredibly fast!

13
Towers of Hanoi (2N) Runtime

• For N 64
• 2N 264 18,450,000,000,000,000,000
• If we had a computer that could execute a million
  instructions per second
• It would take 584,000 years to complete
• But it could get worse

14
The Bounded Tile Problem
Match up the patterns in thetiles. Can it be
done, yes or no?
15
The Bounded Tile Problem
Matching tiles
16
Tiling a 5x5 Area
25 available tiles remaining
17
Tiling a 5x5 Area
24 available tiles remaining
18
Tiling a 5x5 Area
23 available tiles remaining
19
Tiling a 5x5 Area
22 available tiles remaining
20
Tiling a 5x5 Area
2 available tiles remaining
21
Analysis of the Bounded Tiling Problem

• Tile a 5 by 5 area (N 25 tiles)
• 1st location 25 choices
• 2nd location 24 choices
• And so on
• Total number of arrangements
• 25 24 23 22 21 .... 3 2 1
• 25! (Factorial) 15,500,000,000,000,000,000,000,
  000
• Bounded Tiling Problem is O(N!)

22
Tiling (N!) Runtime

• For N 25
• 25! 15,500,000,000,000,000,000,000,000
• If we could place a million tiles per second
• It would take 470 billion years to complete
• Why not a faster computer?

23
A Faster Computer

• If we had a computer that could execute a
  trillion instructions per second (a million times
  faster than our MIPS computer)
• 5x5 tiling problem would take 470,000 years
• 64-ring Tower of Hanoi problem would take 213
  days
• Why not an even faster computer!

24
Where Does this Leave Us?

• Clearly algorithms have varying runtimes.
• Wed like a way to categorize them
• Reasonable, so it may be useful
• Unreasonable, so why bother running

25
Performance Categories of Algorithms

• Sub-linear O(Log N)
• Linear O(N)
• Nearly linear O(N Log N)
• Quadratic O(N2)
• Exponential O(2N)
• O(N!)
• O(NN)

Polynomial
26
Cost and Complexity

• Algorithm complexity can be expressed in Order
  notation, e.g. at what rate does work grow with
  N?
• O(1) Constant
• O(logN) Sub-linear
• O(N) Linear
• O(NlogN) Nearly linear
• O(N2) Quadratic
• O(XN) Exponential
• But, for a given problem, how do we know if a
  better algorithm is possible?

27
The Problem of Sorting

• For example, in discussing the problem of
  sorting
• Two algorithms to solve
• Bubblesort O(N2)
• Mergesort O(N Log N)
• Can we do better than O(N Log N)?

28
Algorithm vs. Problem Complexity

• Algorithmic complexity is defined by analysis of
  an algorithm
• Problem complexity is defined by
• An upper bound defined by an algorithm
• A lower bound defined by a proof

29
The Upper Bound

• Defined by an algorithm
• Defines that we know we can do at least this well
• Perhaps we can do better
• Lowered by a better algorithm
• When I was a boy I knew only about Bubblesort
  with O(N2). Today I am a man I have learned
  Mergesort with O(NlogN). John Wayne in True
  Algorithms 1954

30
The Lower Bound

• Defined by a proof
• Defines that we know we can do no better than
  this
• It may be worse
• Raised by a better proof
• For comparison sorting you can probably prove
  that sorting must have a O(N). I can prove that
  sorting has O(NlogN).

31
Upper and Lower Bounds

• The Upper bound is the best algorithmic solution
  that has been found for a problem.
• Whats the best that we know we can do?
• The Lower bound is the best solution that is
  theoretically possible.
• What cost can we prove is necessary?

32
Changing the Bounds
Lowered by betteralgorithm
Upper bound
Lower bound
Raised by betterproof
33
Open Problems

• The upper and lower bounds differ.

Lowered by betteralgorithm
Upper bound
Unknown
Lower bound
Raised by betterproof
34
Closed Problems

• The upper and lower bounds are identical.

Upper bound
Lower bound
Like sorting.
35
Closed Problems

• Better algorithms are still possible
• Better algorithms will not provide an improvement
  detectable by Big O
• Better algorithms can improve the constant costs
  hidden in Big O characterizations

36
Tractable vs. Intractable

• Problems are tractable if the upper and lower
  bounds have only polynomial factors.
• O (log N)
• O (N)
• O (NK) where K is a constant
• Problems are intractable if the upper and lower
  bounds have an exponential factor.
• O (N!)
• O (NN)
• O (2N)

37
Problems that Cross the Line

• The upper bound implies intractable
• The lower bound implies a tractable
• Could go either way

38
NP-Complete Problems
39
Problems that Cross the Line

• What if a problem has
• An exponential upper bound
• A polynomial lower bound
• We have only found exponential algorithms, so it
  appears to be intractable.
• But... we cant prove that an exponential
  solution is needed, we cant prove that a
  polynomial algorithm cannot be developed, so we
  cant say the problem is intractable...

40
NP-Complete Problems

• The upper bound suggests the problem is
  intractable
• The lower bound suggests the problem is tractable
• The lower bound is linear O(N)
• They are all reducible to each other
• If we find a reasonable algorithm (or prove
  intractability) for one, then we can do it for
  all of them!

41
Example NP-Complete Problems

• Path-Finding (Traveling salesman)
• Map coloring sub-network frequencies
• Scheduling classes, airlines
• Matching (bin packing)
• 2-D arrangement problems
• Planning problems (pert planning)
• First-order Predicate Calculus
• http//en.wikipedia.org/wiki/List_of_NP-complete_p
  roblems

42
Class Scheduling Problem

• With N teachers with certain hour restrictions M
  classes to be scheduled, can we
• Schedule all the classes
• Make sure that no two teachers teach the same
  class at the same time
• No teacher is scheduled to teach two classes at
  once
• No rooms double booked

43
Determinism vs. Nondeterminism

• Nondeterministic algorithms produce an answer by
  a series of correct guesses
• Deterministic algorithms (like those that a
  computer executes) make decisions based on
  information.

44
NP-Complete

• NP-Complete comes from
• Nondeterministic Polynomial
• Complete - Solve one, Solve them all
• There are more NP-Complete problems than provably
  intractable problems.

45
Proving NP-Completeness

• Show that the problem is in NP. (i.e. show that
  there are a finite number of steps, but you
  dont know which to take when.)
• Assume it is not NP complete
• Show how to convert an existing NPC problem into
  the problem that we are trying to show is NP
  Complete (in polynomial time).

46
Why are we telling you all this?

• If you are working on a tough problem and you can
  prove that its NP-Complete you can
• a. Take the rest of the day off.
• b. Luck out and find a solution
• c. Use an approximation algorithm

47
Decidable vs. Undecidable Problems
48
Decidable Problems

• We now have three categories
• Tractable problems
• NP-Complete problems
• Intractable problems
• All of the above have algorithmic solutions, even
  if impractical.

49
Undecidable Problems

• No algorithmic solution exists
• Regardless of cost
• These problems arent computable
• No answer can be obtained in finite amount of time

50
The Unbounded Tiling Problem

• What if we took the tiling puzzle and removed the
  bounds
• For a given number (T) of different kinds of
  tiles
• Can we arrive at an arrangement that will fill
  any size area?
• By removing the bounds, this problem becomes
  undecidable.

51
The Halting Problem

• Given an algorithm A and an input I, will the
  algorithm reach a stopping place?
• loop
• exitif (x 1)
• if (even(x)) then
• x lt- x div 2
• else
• x lt- 3 x 1
• endloop
• In general, we cannot solve this problem in
  finite time.

52
A Hierarchy of Problems

• For a given problem, is there or will there ever
  be an algorithmic solution?
• Problem In Theory In Application
• Undecidable NO NO
• Intractable YES NO
• Tractable YES YES

53
Questions?
54
(No Transcript)