Column Generation - PowerPoint PPT Presentation


PPT – Column Generation PowerPoint presentation | free to view - id: 130bff-NmMwY


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Column Generation


Column generation is a technique that is widely used to solve large scale ... Thirdly, if the primal RMP has a degenerate optimum, the associated dual has ... – PowerPoint PPT presentation

Number of Views:46
Avg rating:3.0/5.0
Slides: 41
Provided by: hqin


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Column Generation

Column Generation
  • ENGG 6070
  • Instructor Shawki M Areibi
  • Students Limin Ma, Hao Qin

What is Column Generation ?
Column generation is a technique that is widely
used to solve large scale integer programs
Prototype of Linear Programs of Large-scale
z Max z c1 x1 c2 x2 … cn xn ,
Subject to ai1x1 ai2 x2 … ainxn
bi ( i 1, 2, …, m) (1)
xn ? 0 ( j 1, 2, …,
n) where n is very large , may be more
than millions.
Example Cutting stock problem A paper
company must produce various sizes of its paper
products from its raw rolls to meet customer
Example Cutting stock problem If customers
need bi rolls of sizes Li , how the company meet
the demand using the fewest number of rolls ?
Example Cutting stock problem A key obstacle
In general, there are exponentially many
patterns xn to cut raw rolls
Structure of Large-Scale Problems
The Decomposition Algorithm
Decompose the large scale system in many
subsystem for computation. For Independent
subsystem , it can be written as Minimize
subject to
bi (i 1, 2, …,t)
bi (i t1, t2, …,u)
bi (i
u1, u2, …,m)
xj gt0 (j 1, 2, …, n)
  • Why Decomposition ?
  • Provide significant computational savings,
    since the computations for linear programs
    are quite sensitive to m, the number of
    constraints, in practice proportionally to m3.
    So it can save 1/m2 times.
  • Each of the independent subproblem can be
    treated separately. Data can be gathered,
    analyzed and stored separately.

Why Column generation ?
  • Because of the large number of variables and
    columns, directly solution by the simplex method
    may be inappropriate. Simply generating all the
    coefficients data aij usually prohibits this
  • Column generation extends the technique
    introduced in the decomposition algorithm, of
    using the simplex method (SM), but generating the
    coefficient data only as needed. Consequently,
    new data could be generated by solving this
    linear program with an appropriate objective

The Approach of Column Generation
As in decomposition, assume a priori that certain
variables, say xJ1, xJ2, …, xn are nonbasic and
restrict their values to zero. The problem
becomes zJ Max c1x1 c2x2 …
cJxJ , (1)
subject to ai1x1 ai2 x2 …
aiJxJ bi ( i 1, 2, …, m) (2)
xj 0 (
j 1, 2, …, J) The original problem (1) is
called a master problem (MP), whereas problem (2)
is called the restricted master problem (RMP).
Suppose that the RMP has been solved by SM
and that ?J1, ?J2, …, ?Jm are the optimal shadow
prices. It is optimal if the simplex optimality
condition holds, that is, if
the original problem has been solved without
specifying all of the aij data or solving the
full MP.
then the simplex method, when applied to
MP, would introduce variable xs into the basis.
The new RMP can be solved by the simplex
method and the entire procedure can be repeated.
The Significance of Column Generation
  • This procedure avoids solving the full MP
    instead it alternately solves a RMP . It is
    referred to as a subprobelm.
  • The method is specified in flow-chart in
    Figure 2. Its efficiency is predicted upon
  • Obtaining an optimal solution before many
    columns have been added to RMP.
  • Being able to solve the subproblem effectively.

(No Transcript)
The column generation method might generate
a non-integer solution. For applications (with
large demands), rounding to nearest integers
produces satisfactory answers. If an exact
optimal is needed, one can apply the BB
Two Major Types of Column Generation
Type I Airline crew scheduling (ACS) Type II
The cutting stock (CS) problem
Two Types of Column Generation
  • Type I column generation
  • Uses an auxiliary model (AM) to
    identify an attractive set of columns,
    defining a RMP that optimizes over these
    explicitly defined columns.
  • The RMP accepts these columns and does
    not interact further with the AM.

Type I Airline crew scheduling (ACS)
  • The objective is to minimize total
    crew cost. Airline schedules give the
    type of aircraft and its departure and arrival
    times for each flight segment, a flight from one
    city to another.
  • A crew works a rotation, which services
    a series of segments and may last several days.
    Each rotation must satisfy a number of
    constraints imposed by official agencies, union
    rules and airline restrictions.
  • The cost of rotation includes lodging and
    other travel expenses, flight time and deadheading

The algorithm of ACS
Defining aij 1 if rotation j ( j?J ) serves
segment i (i ? I ), 0 otherwise and employing
decision variables x j 1 if rotation j is
prescribed, 0 otherwise ( j ? J ), Problem
ACS may be formulated as Min
ZACS (4.1) s.t.
i ? I
x j ?0, 1 j
? J (4.3)
Challenge there may be millions of
feasible rotations for a commercial airline
so that even generating them may require a
prohibitive amount of run time. Solution 1.
Invokes an AM to generate a set of feasible
rotations. Each generated column aj ( j ?
J ) consists of elements aij 1 (i ? Ij )
for each segment i served by
rotation j , and aij 0 (i ? Ij ). 2.
Solves the set partitioning problem with the
columns defined explicitly by the AM.
  • Experience Result
  • It is typically not possible to assure that
    a restricted set of rotations includes a subset
    that comprises an optimal solution, so
    the overall approach is a heuristic.
  • Nevertheless, experience has shown that it
    is possible to prescribe good solutions if
    the set partitioning problem can be solved
    effectively for a large number of generated

Type II column generation
Type II column generation has two classic
applications i) The cutting stock (CS)
problem ii) The vehicle routing problem
The Cutting Stock (CS) Problem.
A company stocks rolls of sheet metal in
standard lengths Lm (m ? M). Assume that the
company has an unlimited number of each
standard length and that will cut these
standard lengths into shorter lengths
to satisfy customers orders. Customers
require bi pieces of length i (i ? I ), where
I is the set of cut lengths ordered. Note
that order I can be satisfied as long as
li ? Lm for some m. Standard length Lm costs
cm(m ? M) and the total cost of filling
orders includes the costs of all standard lengths
cut to fill the orders. The cutting stock
problem is to fill customer orders at minimum
total cost.
The Cutting Stock (CS) Problem.
Let aij denote the number of pieces of length i
(i ? I ) in cutting pattern j and c j , the
cost of the standard roll used in pattern j .
Thus, the index j implies m as well as I j ,
the subset of cut lengths included in the
cutting pattern. To be feasible, cutting
pattern j must observe the limitation imposed
by the standard length Lm(m ? M)
The algorithm
Decision variables x j number of cutting
patterns of type j prescribed ( j ? J ) to
present Problem CS as formulated
(5.1) s.t.
i?I (5.2)
x j ? 0, Integer j ? J.
The algorithm
Linear relaxation for integer constraints The
requirement of the cutting stock problem is known
to be tight. This heuristic is not guaranteed to
prescribe the optimal integer solution. To
guarantee such a solution, the linear relaxation
may be used to obtain a lower bound at each
node in a branch-and-bound (BB) search tree.
The algorithm
The primary difficulty is that the number of
cutting patterns, J , may be extremely large,
so that solving the LP problem with all J
columns could take a prohibitive amount of
The algorithm
Instead of listing all J columns explicitly
and pricing each out to identify the column
that should enter solution at each simplex
iteration, the RMP generates a limited set of
columns and must be optimized over these
columns to obtain dual variables that are used
to define an dual objective function .
Defining wi dual variable associated with
constraint i of type (5) (i . I ) and cm cost of
standard length Lm(m . M) and employing decision
variables yi number of length i in a cutting
pattern j .
The dual solution
It can be formulated as Min

(6.1) s.t.

yi ? 0, Integer i ? I
(6.3) If ZPOP ?0 (m ? M), the current RMP
solution is optimal. Otherwise, if ZPOP ?0, the
dual solution generates an improving column,
which is assigned a new column number j
( j ?j 1), an objective function
coefficient c j cm and coefficient in each row
i (i ? I )aij yi (i ? I ).
Comparison between Type I and Type II
  • The primary limitation of the Type I method is
    that the AM generates columns without interacting
    with the RMP. It normally has to generating a
    large set of columns with the hope of including
    the subset that comprises the optimal solution
  • Type II methods, which do involve interaction
    between the RMP and SP(s), allow columns to be
    identified and introduced as needed

Other Problem Can be Solved by Column Generation
  • Generalized assignment problem
  • The vehicle routing problem
  • Single machine scheduling
  • Assembly system design with tool changes.
  • Problems that can be solved by the
  • decomposition.

Technical considerations
i) Formulation issues a) RMP formulation The
goal is to formulate a decomposition that will
allow the solution to the RMP to serve as a
tight bound on the value of the optimal integer
solution to facilitate the BB search
(1) They may provide tighter bounds than
more compact formulations can (2) They
may avoid symmetry that requires BB to
expend considerable run time to enumerate a
large number of equivalent solutions (3)
They are amenable to decomposition, perhaps
separating into a number of independent SPs that
may be solved easily and (4) They may provide
the only formulation possible.
b). SP formulation SP(s) also play a
crucial role through their structure,
symmetry, complexity and whether of not they
exhibit the Integrality Property. The integer
variables in an IP typically become decision
variables in one or more SPs. Formulations that
have the block diagonal structure are attractive
because they result in small, independent SPs
that are typically more effectively solved.
ii) Algorithm design issues a) Branching
strategies Branching must not destroy SP
structure and should result in siblings that
represent balanced sets of solutions.
Balancing is important because it can be expected
to result in a tighter bound at each sibling
node, facilitating solution. b) Subproblem
solution strategies SP solution strategies select
the column that will enter the RMP and a method
to solve each SP. One strategy would be to solve
all SPs and select the best improving column
to enter the RMP.
iii) Implementation issues a) Pool
management 1. Determine an initial BFS for the
RMP 2. Initialize the column pool to be empty 3.
Optimize the RMP over columns that are known
explicitly 4. Delete nonbasic generated columns
in the RMP that have positive reduced costs 5.
Select a subset of remaining pool columns with
negative reduced costs, add them to RMP and go to
step (3)
6. Eliminate any remaining columns from the
pool 7. Use a heuristic to solve the SP(s), If
improving columns are generated, add them to the
pool and go to step (5) 8. If no columns are
generated in step (7), use an optimizing
algorithm to solve the SP(s) If improving
columns are generated, add them to the pool
and go to step (5) 9. Stop
  • b) Degeneracy
  • Degeneracy in the RMP may be problematic
    for several reasons.
  • First, a number of pivots ( and
    corresponding SP solutions to generate
    entering columns) may be required before the
    search can progress to a better, adjacent
    extreme point.
  • Second, special steps may be required to avoid
    a large number of pivots (called stalling) and
    the worst-case, cycling.
  • Thirdly, if the primal RMP has a degenerate
    optimum, the associated dual has
    alternative optimal solutions and the best
    set of optimal dual variables must be identified
    so that they can be passed into the SPs
    and the primal basic feasible solution that
    satisfies the LP optimality criterion may be

Conclusions and recommendations for future
  • Several issues must be resolved by future
    research to enhance existing capabilities.
  • First, as in any LP problem, degeneracy causes
    inefficiency by requiring a number of pivot
    operations to advance beyond a degenerate
    extreme point and may lead to stalling and
  • Second, SP solution strategies may require a
    large number of SPs to be solved for each
    column that is entered into the solution of the

Thank you !