Chapter 6: IP Techniques

1
Chapter 6 IP Techniques

• Overview
• Enumeration Techniques
• Complete Enumeration list all solutions and
  choose the best
• Branch and Bound Implicitly search all
  solutions, but cleverly eliminate the vast
  majority before they are even searched
• Implicit Enumeration Branch and Bound applied to
  binary variables
• Cutting Plane Techniques Use LP to solve integer
  programs by adding constraints to eliminate the
  fractional solutions.

2
Complete Enumeration

• Systematically considers all possible values of
  the decision variables If there are n binary
  variables, there are 2n different ways.
• Usual idea iteratively break the problem in two.
  At the first iteration, we consider separately
  the case that x1 0 and x1 1.
• Suppose that we could evaluate 1 billion
  solutions per second.
• Let n number of binary variables
• Solutions times
• n 30, 1 second
• n 40, 17 minutes
• n 50, 11.6 days
• n 60, 31 years

3
Branch and Bound

• General idea Divide-and-conquer
• Branching we break a big problem into manageable
  size problems and solve them
• Bounding At each node Si
• 1. Solve the linear program at the node
• 2. Eliminate the subtree (fathom/prune it) if
• Optimality An optimal solution of IPi is found
  the solution is integer (there is no need to go
  further) or
• Value Dominance We can establish that
  ziIP? zIP ( for the max problem ). The best
  solution in the subtree cannot be as good as the
  best available solution (the incumbent) or
• Infeasibility Si ? There is no feasible
  solution

4
Example

• max -100x1 72x2 36x3
• s.t. -2x1x2 ? 0, -4x1x2 ? 0, x1x2x3 ? 1 and
  x ? B3

S
x1 1
x1 0
S0
S1
Pruned because S0 ?
x2 1
x2 0
S11
S10
Pruned by value dominance -10036 lt 8
x3 0
x3 1
S111
S110
z110 -28
z111 8
5
BB Algorithm

• Consider the IP zIP max cx x ? S
• (Initialization) ? IP, S0 S, ,
  and
• (Termination test) If ? ?, then solution x0
  is optimal
• (Problem Selectionrelaxation) Select and remove
  a problem IPi from ? and solve its relaxation
  RPi
• (Pruning) Test if the tree can be pruned at Ipi
• ? then go to Termination test
• go to next step
• , set . Delete from
  ? all problems with . If
  go to step 2 otherwise step 5
• (Division) Add IPijj1K to ?

6
Implementation Issues

• There are a number of unspecified steps in the
  algorithm
• Branching how do we divide a problem into
  subproblems?
• Searching how do we select the next node to
  search?
• Bounding how do we develop the relaxation of a
  problem?

7
Bounding Techniques

• We use the bound obtained by dropping the
• integrality constraints (LP relaxation). There
  are
• Key tradeoff for bounds time to obtain a bound
• If one can obtain a bound much quicker,
• sometimes we would be willing to get a bound
• It usually is worthwhile to get a bound that is
• better, so long as it doesnt take too long

8
Branching

• Branching determining children for a node. There
  are many choices.
• Rule of thumb 1 solution, it is often good to
  branch on xj 0 vs xj 1.The hope is that a
  subdivision with xj 0 can be pruned.
• Rule of thumb 2 branching on important variables
  is worthwhile. e.g., in the location problem,
  branch on the plant location variables first
• Given that we are working with a linear
  relaxation, one possibility is to perform a
  disjunction on Si Si ? x dx ? d0 and Si ? x
  dx ? d01 where (d, d0) is integral
• In practice, only very special choices of (d, d0)
  are used
• Variable dichotomy Choose d ej for some j and
  d0
• GUB dichotomy Assuming the model has
  perform the division with constraints
• and

9
Branching variable

• User specified priorities
• Degradation or penalty to estimate the decrease
  in the upper bound of a node from forcing a
  variable to be integral
• xj
• When we branch on xj, estimate a decrease of
  for one child and for the
  other child
• can be user supplied or
  estimated in the procedure

10
Node Selection

• A priori rules
• Depth-first search with back tracking
• Breadth-first search
• Adaptive rules
• Node with the best upper bound
• Node with the best estimate
• Node that is likely to yield the quickest
  improvement on

11
Branch and Bound Example
A Internet service provider is considering the
purchase of two types of servers (large
computers). The ISP has a budget of 400,000 and
200 square feet of available space at its server
farm. Purchase prices, space requirements, and
expected daily revenues are shown below.
12
BB Formulation
Max Z 1,000x1 1,500x2 s.t. 80,000x1
40,000x2 ? 400,000 15x1 30x2 ? 200 x1, x2 ? 0
and integer
13
BB Solution
14
Cutting Planes Algorithm

• Instead of partitioning the feasible region, the
  (pure) cutting plane technique works with a
  single LP
• It adds cutting planes (valid linear programming
  inequalities) to this LP iteratively.
• At each iteration the feasible region is
  successively reduced until an integer optimal is
  found by solving the LP
• In practice, it is also used as part of branch
  and bound. The essential idea is finding valid
  cuts or inequalities.
• Where do these cuts come from? ? Two approaches
• Problem specific Illustrated on Traveling
  Salesman Problem and Knapsack Problem
• LP-based approach, that works for general integer
  programs Gomory cutting planes

15
Problem Specific Cuts The capital budgeting
(knapsack) problem

• Formulation
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6 14
• xj binary for j 1 to 6
• The LP Relaxation
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6 14
• 0 xj 1 for j 1 to 6
• The optimal solution x1 1, x2 3/7, x3 0,
  x4 0, x5 0, x6 1
• Can we find a valid inequality (cut) that
  eliminates this solution?
• 5x1 7x2 6x6 14 ? x1 x2 x6 2

16
The Knapsack Problem

• After one cut
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6 14
• x1 x2 x6 2
• 0 xj 1 for j 1 to 6
• The optimal solution x1 0, x2 1, x3 ¼,
  x4 0, x5 0, x6 1
• 7x2 4x3 6x6 14 ? ?????
• After two cuts
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6
  14
• x1 x2 x6 2
• x2 x3 x6 2
• 0 xj 1 for j 1 to 6
• The optimal solution x1 1/3, x2 1, x3
  1/3, x4 0, x5 0, x6 1
• 5x1 7x2 4x3 6x6 14 ? ?????

17
The Knapsack Problem

• Obtaining the valid cut it is easy to
• maximize x1 x2 x3 x6
• s.t. 5x1 7x2 4x3 6x6 14
• xj binary for j 1, 2, 3, 6.
• Greedily put the smallest budget items in the
  knapsack until no more can be put in. In this
  case, items 1 and 3 are put in, and there is no
  room for item 2 or 6. So x1 x2 x3 x6 2.
• After three cuts
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6
  14
• x1 x2 x6 2
• x2 x3 x6 2
• x1 x2 x3 x6 2
• 0 xj 1 for j 1 to 6
• Note the new cuts dominates the other cuts.

18
The Knapsack Problem

• Eliminating the redundant constraints
• maximize 16x1 22x2 12x3 8x4 11x5 19x6
• subject to 5x1 7x2 4x3 3x4 4x5 6x6 14
• x1 x2 x3 x6 2
• 0 xj 1 for j 1 to 6
• The optimal solution x1 0, x2 1, x3 0, x4
  0, x5 1/4, x6 1 z 43 ¾ So, z 43
• Summary for knapsack problem
• We could find some simple valid inequalities that
  showed that z 43. This is the optimal objective
  value.
• It took 3 cuts Had we been smarter it would have
  taken 1 cut.
• We had a simple approach for finding cuts This
  does not find all of the cuts.
• Recall, it took 25 nodes of a branch and bound
  tree.
• In fact, researchers have found cutting plane
  techniques to be necessary to solve large integer
  programs (usually as a way of getting better
  bounds.)

19
The Traveling Salesman Problem (TSP)

• Very well studied problem. Its often the problem
  for testing out new algorithmic ideas
• NP-complete (it is intrinsically difficult in
  some technical sense)
• Large instances have been solved optimally (5000
  cities and larger)
• Very large instances have been solved
  approximately (10 million cities to within a
  couple of percent of optimum.)
• We will formulate it by adding constraints that
  look like cuts
• Formulation of the TSP as an IP
• Let xij 1 if arc (i, j) is in the tour, xij
  0 otherwise
• Minimize ?(i,j)?Acijxij
• subject to ?i(i,j)?Axij 1, ?j(i,j)?Axij
  1
• Are these constraints enough? No? if U?V and 2
  U V - 2 then
• ?i?U,j ?V\U,(i,j) ?A xij 1 (subtour breaking
  constraints)
• The subtour elimination constraints are good in
  the sense that for practical problem the LP bound
  is usually 1 to 2 from the optimal TSP tour
  length.
• One can add even more complex constraints, and
  people do. (and it helps)
• Other subtour constraints ?i?U,j ?U,(i,j) ?A xij
  U-1 and Tahas book

20
The Chvatal-Gomory rounding method (C-G method)

• Let Aj denote the jth column of A
• For any u ? Rm, u ? 0,
• x ? 0 implies
• Therefore,
• After integer rounding

21
Example

• Consider P (x1, x2) x12x2 ? 5,
• -x1 ? 0,
• 
  -x2 ? 0.
• Therefore, A1 (1, -1, 0) and A2 (1, 0, -1)
• Using u (0.5, 0, 0), we get 0.5x1x2 ? 2.5
• After integer rounding x2 ? 2

22
An Algorithm for Pure IPs

• Problem max cx Ax ? b, x ? 0, x integer
• Assume A and b have only integer entries, and the
  problem is bounded
• Change inequalities to equalities by adding m
  slack variables
• Max cx Ax Is b, x, s ? 0, x integer
• Solve the linear programming relaxation
• Max cx Ax Is b, x, s ? 0
• If the optimal solution is integral, we are done.
  
• Otherwise, there must be a row in the simplex
  tableau with fractional right hand side. Assume
  its the jth row xj ?, where ? is
  positive fractional.

23
Algorithm for Pure IPs

• We obtain the valid inequality by C-G method
  where uj 1 and all other values are 0
• xj ?
• Add the inequality to the LP and resolve
• The Gomorys Fractional Cutting Plane Algorithm
  (FCPA)
• Repeat
• Solve current linear program
• If the LP optimal solution is fractional
• Identify an LP row with fractional right hand
  side
• Use the row to generate a valid inequality using
  C-G method that cuts the optimal LP
• Until
• Either LP optimal is integer (IP optimal)
• Or LP is infeasible (? IP infeasible)
• Note that the algorithm relies on LP solution to
  be bounded!

24
Example

• Consider the following IP
• maximize z 7x1 10x2
• subject to x1 3x2 6
• 7x1 x2 35
• x1, x2 0 and integer