Applications of knapsack problems

1
Applications of knapsack problems
10 September 2002
2
Applications of knapsack problems

• What the hell is a knapsack?
• Knapsack cover inequalities lifting surrogate
  duality and other boring definitions
• Knapsacks and infeasibility
• Knapsacks and (bounded) general integer variables

3
A knapsack problem is simply an integer programme
with a single constraint

• Max cx
• Subject to
• ax b
• x?Bn
• Suppose you have a knapsack, with size b, and a
  bunch of items with size ai, and value ci, and
  wish to put the most value into the knapsack
• and can somehow mash the items so that they fit
  whenever the sum of their volumes fit

4
they come in several flavours

• Pure binary knapsack
• Max cx
• Subject to
• ax b
• x ? Bn
• Bounded knapsack
• Max cx
• Subject to
• ax b
• 0xu
• x ? Zn

Unbounded knapsack Max cx Subject to ax
b x0 x ? Zn Bounded mixed integer
knapsack Max cxdy Subject to axzy
b x0 x ? Zn y ? Rm
Bounded knapsacks are our focus here, though most
results are or appear to be readily generalisable
to bounded mixed integer knapsacks
5
and can be used to solve all integer
programming problems (if you know the right
multiplier)

• Surrogate Duality
• Maxx cx
• Subject to
• Ax b
• x ? Bn
• Minu
• (Maxx cx
• Subject to
• uAxub
• x ? Bn )
• u ? Rn

• Any integer programme with many constraints
• can be solved by finding the right knapsack
  problem.
• uAxub is called a surrogate knapsack
• trouble is, finding the u that minimises this
  problem is difficult, and (a common drawback with
  dual approaches) the solution to the inner
  problem is not feasible at any stage until we
  find an optimal u

6
in psuedo-polynomial time!

• Solving a knapsack problem in pseudo-polynomial
  time
• Dynamic Progamming
• Let KP(n,b)
• Maxx c1x1c2x2cnxn
• s.t.
• a1x1a2x2anxnb
• Now
• KP(0,b)0 (Base Case 1)
• KP(k,l), llt0 -8 (Base Case 2)
• KP(n,b) max(KP(n-1,b), cnKP(n-1,b-an))
  (Recursive case)
• So, assuming the KP is scaled such that the ai
  are integral, the worst case solution time is
  proportional to nb.

7
or can at least help solve them (when you
cant find the right multiplier)

• Finding all the facets (almost never pseudo-
  polynomial!) seems insane given we can just solve
  uAxltub by dynamic programming
• HOWEVER .
• Its hard to find the optimal multiplier u - and
  because its a dual method, if we dont find u
  we dont even have a feasible solution to the
  primal problem.
• Often its easy to find a probably fairly close
  to optimal multiplier u (Well see some
  approaches shortly)
• If u is close to u then we expect
• Some of the facets of u and u may coincide
  even getting some of the facets may cut off
  fractional values in LP based branch-and-bound
• Facets of u to coincide with facets of u in
  many dimensions

• Surrogate duality insane method
• Find the right multiplier u.
• Generate all the facets (Gxg) of
• uAxub
• x ? Bn
• Solve the linear programme
• max cx
• Subject to
• Gxg
• x ? Bn

8
Whats a valid inequality/facet/dimension?
9
inside a Branch-and-bound-and-cut-and-preprocess
-and-probe-and-run-a-primal-heuristic-and-variable
-fix-and-whatever algorithm

• Surrogate cutting planes
• Inside a branch-and-bound code
• Find a half decent multiplier u.
• Generate a facet (bxb0) of
• uAxub
• x ? Bn
• Solve the linear programme
• max cx
• Subject to
• Axb
• bxb0
• x ? Bn
• Repeat so long as the bxb0s are doing a
  reasonable job of cutting of fractional LPs.

• The remainder of our discussion will focus on
• How to find a half decent multiplier
• How to Generate a facet

10
The facets (or high dimension inequalities) we
are interested in are called (lifted) knapsack
cover inequalities.
For binary spaces

• A cover C is a set of items that dont fit into
  the knapsack.
• A minimal cover C is a cover that only just
  doesnt fit i.e. every strict subset of those
  items will fit, but the set C wont.
• A Knapsack cover inequality (KCI) simply states
  that therefore one of the items must not be in
  the knapsack.
• If C is minimal then the KCI is facet-defining
  for the space
• Lifting can then be use to come up with
  coefficients for the other variables so that we
  get a facet of the full space

11
Generation of Knapsack Cover inequalities is
frequently applied to solve sparse binary
problems

• An integer programme where each variable occurs
  in only one constraint is separable into the
  individual knapsack constraints.
• A
• A sparse constraint matrix is typically close to
  separable e.g.
• A
• Essentially this suggests that u(1,0,0,0,0),
  u(0,1,0,0,0,) etc. are pretty reasonable
  choices for u.

Just solve each of the separate knapsack problems
to solve the overall integer programme
Here we cant just solve the separate knapsack
problems, but a facet for any of the knapsacks is
valid for the whole problem, and because there is
little overlap, is likely to be either facet
defining, or of high dimension for the whole
problem.
12
and has been very successful at reducing the
computational effort required
13
Knapsacks and infeasibility

• Most cutting plane algorithms generate cuts
  aiming to cut off the current LP solution at a
  feasible node of the branch and bound tree
• but thats boring- we can also generate useful
  cutting planes at infeasible nodes
• or nodes being pruned
• and get a reasonable performance boost

14
Most cutting plane algorithms generate cuts
aiming to cut off the current LP solution at a
feasible node of the branch and bound tree

• Cut separation- e.g. Binary knapsack cover
  inequalities
• Given the current LP solution xlp
• Find the minimal Cover C which maximises
• (i.e. maximise the amount that the KCI is
  violated by the current LP)
• Then lift it.

15
but thats boring- we can also generate useful
cutting planes at infeasible nodes

• And with some trickery based on the properties
• rb-z.1 lt 0 (improves objective)
• rA-zIB? 0 (definition of a ray)
• of a divergent ray, we find that
• Bz B ? i zi?0
• is a cover for the surrogate knapsack rAx rb
• rAx ? zIBx, for any x ? 0 (ii)
• ? z.1 for any x xi1, i ?Bz (i)
• gt rb

• Recall an infeasible primal ? unbounded dual ? a
  divergent ray (r,z) exists for the current LP
• Min cx
• s.t.
• Axb
• xi1, i?S1 (these constraints added by branching.
  W.lo.g. we assume all the branches were
  1-branches)
• r is the multiplier for the original
  constraints, z for the branching constraints

16
or nodes being pruned
So, at an infeasible node, theres no LP were
trying to cut off-how can this approach help?!?
Example After adding four branching
constraints x11, x21, x31, x41 An infeasible
node is found. The divergent dual ray (r,z) has
z(0,0,23,12), so Bz3,4 And the constraint
x3x41 is added to the formulation. This does
nothing useful at the node that causes it to be
added. but it does stop us exploring nodes with
branching constraints x10, x21, x31,
x41 x11, x20, x31, x41 x10, x20, x31, x41

• Let zBEST be the best integer solution weve
  found so far.
• Temporarily add the constraint
• cxzBEST
• to the formulation at nodes being pruned.
• now they are infeasible and we can apply the
  previous method.
• Actually implementation-wise its quite
  different, as wed rather not keep pivotting
  until the LP solver proved it was infeasible with
  the new constraint

17
and get a moderate performance boost
18
Knapsacks and (bounded) General Integer Variables

• Whilst you can always convert general integer
  variables to binary many of us are not that
  patient.
• Ceria et. al. made a start generalising KCI to
  general integers
• Extending this approach with some much less
  natural generalisations
• Ceria constraints arent necessarily facet
  defining for the cover space
• but lifted SMD subcovers are
• On general integers minimal is sometimes too
  small
• so we redefine minimal to be less small
• and get significant computational improvements

19
Whilst you can always convert general integer
variables to binary many of us are not that
patient.
20
Ceria et. al. made a start generalising KCI to
general integers.
BOUNDED INTEGER (Ceria et. al.)
BINARY

• A cover C is a set of items that dont fit into
  the knapsack.
• A minimal cover C is a cover that only just
  doesnt fit i.e. every strict subset of those
  items will fit, but the set C wont.
• A Knapsack cover inequality (KCI) simply states
  that therefore some of one of the items must not
  be in the knapsack.
• If C is minimal then the KCI is facet-defining
  for the space Pblah
• Lifting can then be use to come up with
  coefficients for the other variables so that we
  get a facet of the full space Pblah

? ?
??
21
but they arent facet defining for the cover
space

• Example
• Consider
• 3x15x27x3 50, 0 xi 4
• C1,2,3 is the only cover (12 20 28 60 gt
  50 l10
• The Corresponding Ceria inequality is
• x1x2x3 10
• (4,4,2), (4,2,4) (2,4,4) are 3 affinely
  independent points that satisfy this at equality.
  However only (4,4,2) and (4,2,4) are feasible
  this constraint is not facet defining.

22
but we can fix that with Strong minimal
dependence.

• Should C not be SMD
• Project out variables by fixing them to their
  upper bounds, until it is
• 3x15x27x3 50, 0 xi 4
• 1,2,3 is a cover, but not SMD.
• Project out x1
• 5x27x3 50-1248, 0 xi 4 l10
• 2,3 is SMD (k2) for the reduced space.
• x2x3 6 is a facet for the reduced space both
  (2,4) and (4,2) lie on it and are feasible)
• By applying lifting to get a constraint for the
  original space
• 0.5x1x2x3 8
• which is facet defining- (0,4,4), (4,2,4)
  (4,4,2) lie on it
• The SMD condition tells us when we need to
  project and then lift in order to get a facet

• A Cover C is Strongly minimally dependant (SMD)
  if (additionally) for some k
• BUT
• The Ceria inequality is facet defining iff the
  cover C happens to be SMD

23
and sometimes minimal coves are too small
worse still- for some knapsack/LP solution
combinations all minimal covers have this
problem Example u(6,4), xlp(2,2.75) 6x14x22
3 1 is the only minimal cover. The
corresponding KCI x13 does not cut off the
current LP solution.

• Problem
• Any minimal cover with
• cannot ever lead to a KCI that cuts off xlp, as
• is a constraint in the current LP, and clearly

24
we can (often- but not always) fix that with
Ceil(x) minimality

• Solution- Define a more complicated
  generalisation of minimality
• A cover is Ceil-x-minimal if
• and
• and
• (Make the cover too big however and no SMD
  subcover exists)
• in binary spaces all ceil-x-minimal covers are
  also minimal covers so this issue does not arise.
• Occasionally no ceil-x-minimal cover exists- and
  therefore no violated KCI

25
and get significant computational improvements