Foundation of Computing Systems

1
Foundation of Computing Systems

• Lecture 20
• Classic Optimization Problems

2
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

3
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

4
Coin Changing Problem

5
Coin Changing Problem

• Problem Definition
• Given a set of denomination of the coins, say
  c1, c2, c3, , cn, such that c1 gt c2 gt c3 gt gt cn
  1.
• Given an amount A, the problem is to
  determine the minimum number of coins needed to
  make A in change.
• Note that
• The problem may assume cn1, so that it is always
  possible to make change for any amount A.
• Any number of coins for each denomination is
  available.

6
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

7
Optimal Retrieval Problem
???
8
Optimal Retrieval Problem

• Problem Definition
• Suppose, that n programs having length l1,
  l2, , ln are stored on a tape. If the programs
  are stored in the order li1, li2, , lin, the
  time to retrieve any program ik is
• (We assume that retrieval time starts at the
  beginning of the tape. The above retrieval time
  for a program ik results from the fact that in
  order to retrieve program ik, we must pass all of
  the programs stored before program ik).
• The average retrieval time is defined as
• The problem is to find an ordering of the
  program to minimize the average retrieval time.

9
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

10
Traveling Salesman Problem
11
Traveling Salesman Problem

• Problem definition
• A salesman has a list of cities, each of
  which he must visit exactly once. There are
  direct roads between each pair of cities on the
  list. The problem is to find route the salesman
  should follow so that he travels the shortest
  possible distance on a round trip , starting at
  any one of the cities and then returning there.
  He should not travel a city twice except the
  starting city.

12
Different Versions of Traveling Salesman Problems

• Hamiltonian Cycle Problem
• This problem is to find Hamiltonian cycle
  with minimum weight graph in graph theory and
  stated as below.
• Given an connected weighted graph and it is
  required to find a tour (a cycle through all the
  vertices exactly once except the first ) of
  minimum weight.
• Euclidian Traveling Salesman Problem
• This problem is to determine the shortest
  path closed tour that connects a given set of
  n-points on a plane.
• Bitonic Tour Problem
• A tour starts at the left most point, go
  strictly left to right, and then go strictly
  right to left and finally back to the starting
  point. The objective function and constraint is
  same as in the Traveling Salesman Problem.

13
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

14
Knapsack Problem
15
Knapsack Problem

• The knapsack problem can be posed as follows.
• A thief robbing a store finds n items the i-th
  item is worth vi and weight wi. The thief would
  take as valuable as possible but he can carry at
  most W weights in his knapsack. Which items
  should the thief take?

16
Knapsack Problems

• The knapsack problem can be generalized and
  formally stated as follows.
• Given n items and a knapsack. The knapsack has a
  capacity W. Any item say i, has a weight wi and a
  value vi per unit. If a quantity xi of the item i
  is placed into the knapsack, then a profit vi.xi
  is earned.
• The objective is to obtain a filling of the
  knapsack that maximizes the total profit earned.
• Since the knapsack capacity is W, we require the
  total weight of all chosen object to be at most
  W.
• Formally the problem may be stated as follows.

Subject to
Maximize
17
Different Versions Knapsack Problems

• Discrete Knapsack Problem
• Here, all vis, wis and W are integers and each
  item must be taken or left behind.
• This problem is also called integer knapsack,
  and 0-1 knapsack problem
• Continuous Knapsack Problem
• In this version of the knapsack problem, we can
  take any quantity (i.e. a fraction) of each item
  (rather that having to make a binary (0 or 1)
  choice for each item.)
• Note
• You can think 0-1 knapsack problem as being like
  a gold ingot while an item in continuous problem
  is more like gold dust.

18
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

19
Some Well Known Problems

• Some problems are well known to learn designing
  algorithms
• Sorting
• Searching
• Coin Changing
• Optimal Retrieval Problem
• Traveling Salesman Problem
• Knapsack Problem
• Bin Packing Problem
• Minimum Spanning Tree Problem
• Shortest Path Problem
• Huffman Tree Problem
• Optimal Binary Search Tree Problem

20
Bin Packing Problem
21
Bin Packing Problem

• Problem definition
• Given n items of sizes s1, s2, , sn. All sizes
  satisfy 0ltsi 1.
• The problem is to pack these items in the fewest
  number of bins, given that each bin has unit
  capacity.