Chapter 12 Integer Programming

1
Chapter 12Integer Programming

• by Dr. Peitsang Wu
• Department of Industrial
• Engineering and Management
• I-Shou University

2
Integer Programming

• The mathematical model for integer programming is
  the linear programming problem with the one
  additional restriction that the variables must
  have integer values.
• If only some of the variables are required to
  have integer values, this model is referred to as
  mixed integer programming (MIP).
• IP problem that contain only binary variables
  sometimes are called binary integer programming
  (BIP).

3
Example

• Max
• s.t.
• or

(binary)
4
Some Perspectives

• IP problem with a bound feasible region are
  guaranteed to have just a finite number of
  feasible solution.

5
Some Perspectives

• Some misunderstanding of solving IP
• (i) Having a finite number of feasible solutions
  ensures that the problem is readily solvable.
• ? wrong n10, 210 solutions
• n20, 106 solutions
• n30, 230 109
  solutions
• (ii) Removing some feasible solutions
  (non-integer ones) from a LP problem will make it
  easier to solve. ? wrong The B.F.S. are there
  to guarantee the C.P.F. solution that is optimal
  for the overall problem. This is the key to the
  remarkable efficiency of the simplex method.

6
Some Perspectives

• Most successful algorithms for IP incorporate
  simplex method as much as they can by relating
  portions of the IP problem under consideration to
  the corresponding LP problem. It is referred to
  as its LP relaxation.

7
Some Perspectives

• Some special cases guarantee integer optimal
  solution.
• minimum cost flow problem
• transportation problem
• assignment problem
• transshipment problem
• shortest-path problem
• maximum flow problem

8
Some Perspectives

• Two primary determinants of computational
  difficulty for an IP problem.
• the number of integer variables
• any special structure in the problem.

9
Some Perspectives

• IP is much more difficult to solve than LP
  problems, sometimes it is tempting to use the
  approximate procedure of simply applying the
  simplex method to the LP relaxation and then
  rounding the non-integers values to integer in
  the resulting solution.
• Two pitfalls about the rounded solutions
• What about the optimal solution in LP is not
  feasible any more after rounded?
• No guarantee that the rounded solution will be
  the optimal integer solution.

10
Some Perspectives

• A better approach for dealing with IP problems
  that are too large to be solved exactly is to use
  one of the available heuristic algorithms. These
  algorithm are extremely efficient for large
  problems, but they are not guarantee to find an
  optimal solution.

11
Some Perspectives

• For IP problems that are small enough to be
  solved to optimality, a considerable number of
  algorithms now are available. But there are not
  comparable to the simplex method in terms of
  efficiency. The most popular one is the
  branch-and-bound technique (?????) and implicitly
  enumerate the feasible integer solution. (???)

12
The Branch-and-Bound Method

• We can solve the pure IP problem by some kind of
  enumeration procedure for finding optimal
  solution. But when the number is large, it wont
  work.
• The basic concept underlying the Branch-and-Bound
  (B and B) technique is to divide and conquer.
• ? We can divide the original large problem into
  serial smaller subproblems until the
• subproblems can be conquer.

13
The Branch-and-Bound Method

• The dividing (branching) is done by partitioning
  the entire set of feasible solutions into smaller
  and smaller subsets.
• ? The conquering (fathoming ??) is done partially
  by bounding how good the best solution in the
  subset can be and then discarding the subset if
  its bound indicates that it cannot possibly
  contain an optimal solution for the original
  problem.

14
B and B in BIP Example

• Max
• s.t.

15
B and B in BIP Example

• ?Branching
• Letting x1 0 the resulting subproblem 1 is
• Max
• s.t.

16
B and B in BIP Example

• Letting x1 1 the resulting subproblem 2 is
• Max
• s.t.
• solution tree Solve the LP relaxation of the
  whole
• problem (x1, x2, x3, x4)
  ( , , , )
• Z , therefore Z?
  ,rounded
• we get Z ? bound
  for the whole
• problem.

17
B and B in BIP Example

• Bounding
• Solve the LP relaxation of subproblem 1
• (x1, x2, x3, x4) ( , , , ) Z ,
  therefore we get Z ? bound for the subproblem
  1.
• Solve the LP relaxation of subproblem 2
• (x1, x2, x3, x4) ( , , , ) Z ,
  therefore
• Z ? , rounded, we get Z ? bound for
  the subproblem 2.

18
B and B in BIP Example

• Fathoming
• Since subproblem 1 Z9 , we already get the
  optimal solution for subproblem 1. It is stored
  as the first incumbent (the best feasible
  solution found so far) for the whole problem.
• We continue to fathom subproblem 2.
• Z?16, but may be fathomed by its descendants
  (by creating new smaller subproblems on this
  subproblem). Furthermore, as new incumbents with
  larger values of Z are found, it will become
  easier to fathom in this way.
• Or the third way of fathoming is If the simplex
  method finds that a subproblems LP relaxation
  has no feasible solution then the subproblem
  itself must have no feasible
• solutions. So it can be dismissed
  (fathoming).

19
B and B in BIP Example

• Fathoming
• Summary of fathoming tests
• Test1 Its bound ? Z
• Test2 Its LP relaxation has no feasible
  solutions
• Test3 The optimal solution for it LP
  relaxation is integer. (If this solution is
  better then the incumbent, it becomes the new
  incumbent, and test 1 is applied to all
  unfathomed subproblems with the new larger Z.)

20
Summary of the BIP B and B

• Initialization Set Z -8
• Apply the bounding step, fathoming step, and
  optimality test described below to the whole
  problem.
• If not fathomed, classify this problem as the one
  remaining "subproblem" for performing the first
  full iteration below.

21
Summary of the BIP B and B

• Iteration
• Step1 Branching
• Among the remaining (unfathomed) subproblems,
  select the one that was created most recently.
  (Break ties according to which has the larger
  bound) Branch from the node for this subproblem
  to create two new subproblems by fixing the next
  variable (the branching variable) at either 0 or
  1.
• Step2 Bounding
• For each new subproblem, obtain its bound by
  applying the simplex method to its LP relaxation
  and rounding down the value of Z for the
  resulting optimal solution.

22
Summary of the BIP B and B

• Step3 Fathoming
• For each new subproblem, apply the three
  fathoming tests summarized above, and discard
  those subproblems that are fathomed by any of the
  tests.
• Optimality test
• STOP, when there are no remaining
  subproblems the current incumbent is optimal.
  Otherwise, return to perform another iteration.

23
Ex. (continue) Iteration 2

• Subproblem 3 Fix x11, x20 the resulting
  subproblem is
• Max
• s.t
• is (x1, x2, x3, x4) ( , , , ) Z
  
• Bound of subproblem 3 is

24
Ex. (continue) Iteration 2

• Subproblem 4
• Fix x11, x21 the resulting subproblem
  is
• Max
• s.t.
• LP relaxation of subproblem 4
• ? is (x1, x2, x3, x4) ( , , , )
  Z
• ? Bound of subproblem 4 is

25
Ex. (continue) Iteration 2

• Both bound ? Z 9 ? test 1 failed
• feasible ? test 2
  failed
• noninteger solution ? test 3 failed

26
Ex. (continue) Iteration 3

• Subproblem 5 fix x11, x21, x30
• Max
• s.t.
• LP relaxation of subproblem 5
• ? is (x1, x2, x3, x4) ( , , , )
  Z
• Bound Z ?

27
Ex. (continue) Iteration 3

• Subproblem 6 fix x11, x21, x31
• Max
• s.t
• subproblem 6 fathomed by test 2
• no feasible solution but subproblem 5 is not.

? no feasible solution
28
Ex. (continue) Iteration 3
29
Ex. (continue) Iteration 4

• Subproblem 7 x11, x21, x30, x40
• Z , (x1, x2, x3, x4) ( , , ,
  )
• Subproblem 8 x11, x21, x30, x41
• Z infeasible
• fathomed by test 3, Z 9 ? Z ,
• (x1, x2, x3, x4) ( , , , )

30
Ex. (continue) Iteration 4
31
B and B Solve Pure IP Problem

• Max
• s.t

32
B and B Solve Pure IP Problem

• The relaxation of this IP problem is (x1,x2)(
  , ) Z

33
Ex. (continue) Iteration 1

• subproblem1

34
Ex. (continue) Iteration 1

• ? optimal solution of relaxation LP of sub1 Z
  , x1 , x2

35
Ex. (continue) Iteration 1

• subproblem2

36
Ex. (continue) Iteration 1

• ? optimal solution of relaxation LP of sub2 Z
  , x1 , x2

37
Ex. (continue) Iteration 1

• subproblem2 fathomed
• Z39,(x1,x2)(3,3)
• subproblem1 failed
• ? Z39

38
Ex. (continue) Iteration 2

• subproblem3

39
Ex. (continue) Iteration 2

• ? optimal solution of relaxation LP of
  sub3infeasible
• ? fathomed F(2)

40
Ex. (continue) Iteration 2

• subproblem4

41
Ex. (continue) Iteration 2

• optimal solution of relaxation LP of sub4
• Z , x1 , x2

42
Ex. (continue) Iteration 3

• subproblem5
• optimal solution of relaxation LP of sub5
• Z , x1 , x2
• ? fathomed F(3) , Z

43
Ex. (continue) Iteration 3

• subproblem6
• optimal solution of relaxation LP of sub6
• Z , x1 , x2
• ? fathomed F(1) Z? Z40

44
Ex. (continue) Iteration 3
45
Knapsack Problem

• (BIP with single constraint)

46
Greedy Algorithm

• Compute
• select those with greatest value, assign
  resource to that variable
• if xi? the last variable go to 2,
• else stop.

47
Example
48
Ex. (continue)

• Since the largest value is 2 and assign x71
• Since the largest value is and assign x21

49
Ex. (continue)

• Since the largest remaining is 1 and assign
• arbitrary x41

50
Ex. (continue)

• Since the largest remaining is 1 and assign
• x1 (since only 10 unit left ) ? Z160
• 
  
• is the optimal solution of the relaxation of
  our example problem.
• H.W
• Use Branch and Bound to solve the Knapsack
  problems.

51
Combinatorial Optimization Problems

• A combinatorial optimization problem is any
  optimization problem that has a finite number of
  feasible solution.
• Ex1. (machine scheduling problem)
• Ten jobs must be processed on a single
  machine. You know the time it takes to complete
  each job and the tine at which job must be
  completed (the jobs due date). What ordering of
  the jobs minimize the total delay of the ten
  jobs?

52
Combinatorial Optimization Problems

• Ex2. (traveling salesman problem)
• A sales man must visit each of ten cities.
  Once before returning to his home. What order of
  visiting the cities minimizes the total distance
  the sales man must travel before returning home?
• Ex3. (eight queens problem)
• Determine how to place eight queens on a chess
  board so that no queen can capture any other
  queen.

53
Example

• Branch and Bound approach for machine scheduling
  problem
• Four jobs must be processed on a single machine.

54
Example

• 1?2?3?4
• Total delay16

55
Let
? Optimal 2-1-3-4
56
Cutting Plane Algorithm

• Optimal Tableau for LP relaxation of about
  problem

57
Cutting Plane Algorithm

• Choose any constraint in the LP relaxations
  optimal tableau in which a basic variable is
  fractional. (arbitrarily)
• x1- s1 s2 .?
• define xthe largest integer less than or
  equal to x thus any number x can be written
  xxf, where 0? f ?1.
• become x1- s1 s1 s2 s2
• x1- s1 s2- -
  s1- s2.?
• now add ? - s1- s2 ? 0
• This constraint called a cut.

58
Properties

• Any feasible point for the I.P. will satisfy the
  cut .
• The current optimal solution to the L.P
  relaxation will not satisfy the cut.
• ?cut ?cut off the current optimal solution to
  the L.P. relaxation but does not cut any feasible
  solution to the I.P.
• If the new problem(old cut constraint), has
• integer solution, we get the optimal solution.
• Otherwise, we add a new cut to current problem.
• And continue the process, until we get an
  optimal
• solution.

59
Theorem

• The process of cutting plane algorithm will
  guarantee an optimal solution to the I.P. after a
  finite number of cuts.

60
Ex. (continue )

• We now add ? 0.75-0.75s1-0.25s2?0 to out
  problem the L.P relaxations tableau is

61
Ex. (continue )

• ?optimal tableau
• ?(x1,x2)( , ) is an optimal solution to
  our I.P
• problem. Optimal value is Z

62
Summary of the cutting plane algorithm

• Step 1 Find the optimal tableau for the I.Ps
  L.P relaxation. If all the variables in the
  optimal solution assume integer values, we have
  found an optimal solution to the I.P. Otherwise,
  processed step2.
• Step 2 Pick a constraint in the L.P relaxation
  tableau whose right-hand-side has the fractional
  part close to 1/2. This constraint will be used
  to generate a cut.

63
Summary of the cutting plane algorithm

• Step 2a For the constraint identified in step2,
  write the constraints r.h.s. and each variables
  coefficient in the form xf , where 0? f ?1.
• Step 2b Rewrite the constraint used to generate
  the cut as all terms with integer
  coefficientsall terms with fractional
  coefficients. Then the cut is
• all terms with fractional coefficients
  ?0.

64
Summary of the cutting plane algorithm

• Step 3 Use the dual simplex to find the optimal
  solution to the L.P. relaxation, with the cut as
  an additional constraint. If all variables assume
  integer values in the optimal solution. We have
  found an optimal solution to the I.P.
• Otherwise, we pick the constraint with the
  most fractional r.h.s. and use it to generate
  another cut, which is added to the tableau.
  Continue the process until find an optimal
  solution. This optimal solution will be the
  optimal solution to the I.P.