ESI 4313 Operations Research 2

1
ESI 4313Operations Research 2

• Dynamic Programming 1

2
Dynamic Programming

• Dynamic programming is a technique for solving
  certain types of optimization problems
• The idea is to break up a large, complex problem
  into many smaller, much easier ones
• Usually, this technique can be applied to
  problems in which a sequence of decisions over
  time needs to be made to optimize some criterion

3
Dynamic Programming

• In many cases, solving a problem by dynamic
  programming means
• formulating this problem as a shortest path
  problem in an acyclic network
• The art of dynamic programming lies in how to
  construct this network!

4
ExampleTravel from coast to coast

• You currently live in NYC (1), but plan to move
  to LA (10)
• You will drive
• To save money, you will spend each night of your
  trip at a friends house
• You structure your potential stopovers as
  follows
• In 1 day you can reach Columbus (2), Nashville
  (3), or Louisville (4)
• On the 2nd day, you can reach Kansas City (5),
  Omaha (6), or Dallas (7)
• On the 3rd day, you can reach San Antonio (8) or
  Denver (9)
• On the 4th day, you can reach LA
• To minimize your gas expenses, you are looking
  for the route of minimum length

5
ExampleTravel from coast to coast

• We can classify the cities as follows
• Call all cities that you can be in at the
  beginning of your nth day the Stage n cities
• The idea of solving this problem by dynamic
  programming is to start by solving easy problems
  that will eventually help you solve the entire
  problem
• In particular, we will work backward

6
ExampleTravel from coast to coast

• Denote the distance between city i and city j by
  ci,j
• If city i is a stage t city, we denote the length
  of the shortest path from city i to LA by ft(i)
• Clearly, we would like to find f1(1)

7
ExampleTravel from coast to coast

• First, find the shortest path to LA from each of
  the cities from which you can reach LA in 1 day
  the stage 4 cities
• Note that these problems are trivial, since in
  each case theres only 1 way to go to LA
• More formally,
• f4(8) c8,10
• f4(9) c9,10

8
ExampleTravel from coast to coast

• Then, find the shortest path to LA from each of
  the stage 3 cities
• Note that this means that you should first go to
  a stage 4 city, and then use the shortest path
  from this stage 4 city to LA
• These problems are not as trivial as the first
  ones, but by simply looking at all possible city
  4 problems and the solutions to the first set of
  problems this remains relatively easy

9
ExampleTravel from coast to coast

• From each stage 3 city
• go to a stage 4 city, and then use the shortest
  path from this stage 4 city to LA
• So, for example, f3(5) is equal to
• c5,8 f4(8), or
• c5,9 f4(9)
• Since were interested in the shortest path, we
  have
• f3(5) minc5,8 f4(8) , c5,9 f4(9)

10
ExampleTravel from coast to coast

• Perform the same procedure for the stage 2 cities
• Perform the same procedure for the stage 1 city,
  NYC
• From NYC you should first go to a stage 2 city,
  and then use the shortest path from this stage 2
  city to LA
• We can find the best route from NYC to LA by
  considering all possible stage 2 cities

11
ExampleTravel from coast to coast

• In general, in stage t we are interested in
  finding ft(i) for all stage t cities i
• Using the earlier approach, we can write
• ft(i) minj j is a stage t1 city ci,j
  ft1(j)
• for all stage t cities i

12
Computational efficiency of dynamic programming

• In the example, we could simply enumerate all
  possible paths from NYC to LA
• It is easy to see that there are 3x3x218 paths
• However, suppose that we have more options
• Starting city is again stage 1
• 5 cities in each of 5 stages (stages 2,,6)
• Destination city is stage 7
• Then there are 553,125 paths
• Determining the length of each of these paths
  takes a total of 5x55 15,625 additions and
  3,124 comparisons

13
Computational efficiency of dynamic programming

• How much work is the dynamic programming
  algorithm?
• The stage 6 problems are trivial
• Each of the other problems require
• 5 additions (potential choices for next city to
  visit) and 4 comparisons
• For a total of 4x5x5 5 105 additions and
  4x5x4 4 84 comparisons

14
Characteristics of dynamic programming

• The problem should have stages
• Each stage corresponds to a point at which a
  decision needs to be made
• Each stage should have a number of associated
  states
• The state contains all information that is needed
  to make an optimal decision for the remaining
  problem

15
Characteristics of dynamic programming

• The decision chosen at each stage describes how
  the state at the current stage is transformed in
  the state at the next stage
• The optimal decision at the current state should
  not depend on previously visited states or
  previous decisions
• This is called the principle of optimality

16
Characteristics of dynamic programming

• There must be a recursion that relates the cost
  or reward for stages t, t1, , T to the cost or
  reward for stages t1, t2, , T
• This recursion formalizes the procedure of
  working backwards from the last stage to the
  first stage

17
Dynamic programming formulation

• Stages t 1,,5
• States city
• Decision in each stage
• Choose the stage t1 city to go to
• Dynamic programming recursion
• f4(i) ci,10 for all stage 4 cities i
• ft(i) minj j is a stage t1 city ci,j
  ft1(j)
• for all stage t cities i

18
Dynamic programming without stages

• You must drive from Bloomington to Cleveland
• You are interested in the route that takes the
  least amount of time

19
Dynamic programming without stages
Gary
Toledo
Cleveland
3 hours
3 hours
1 hour
2 hours
3 hours
Dayton
Indianapolis
Columbus
2 hours
3 hours
1 hour
2 hours
2.5 hours
Bloomington
Cincinnati
3 hours
20
Resource allocationthe knapsack problem (1)

• Stockco is considering 4 investments
• Investment 1 will yield a NPV of 16K, but
  requires a cash outflow of 5K
• Investment 2 will yield a NPV of 22K, but
  requires a cash outflow of 7K
• Investment 3 will yield a NPV of 12K, but
  requires a cash outflow of 4K
• Investment 4 will yield a NPV of 8K, but
  requires a cash outflow of 3K
• You have a budget of 14K

21
Resource allocationthe knapsack problem (1)

• IP formulation

22
Resource allocationthe knapsack problem (2)

• You are planning an overnight hike, and are
  considering taking 4 items along on your trip
• Item 1 yields a benefit of 16, but weighs 5 lbs
• Item 2 yields a benefit of 22, but weighs 7 lbs
• Item 3 yields a benefit of 12, but weighs 4 lbs
• Item 4 yields a benefit of 8, but weighs 3 lbs
• You do not want to carry more than 14 lbs
• You want to maximize your benefit
• Mathematically, this is the same problem as the
  investment problem!