Chapter%203%20Linear%20Programming%20Methods - PowerPoint PPT Presentation

View by Category
About This Presentation



( ) Chapter 3 Linear Programming Methods Whereas the simplex method searches for the optimal solution at the boundary of the feasible region, the ... – PowerPoint PPT presentation

Number of Views:64
Avg rating:3.0/5.0
Slides: 72
Provided by: IBT5
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Chapter%203%20Linear%20Programming%20Methods

  • ??????(?)

Chapter 3 Linear Programming Methods

  • Whereas the simplex method searches for the
    optimal solution at the boundary of the feasible
    region, the latest developments are based on the
    idea of following a path through the interior of
    the feasible region until the optimum is reached.
    These so-called interior point methods are of
    great theoretical importance because they provide
    a bound on the computational effort required to
    solve a problem that is a polynomial function of
    its size.
  • No polynomial bound is available for the simplex
    algorithm. Nevertheless, simplex codes have
    proven to be highly efficient in practice and
    remain at the center of virtually all commercial
    optimization packages.


Standard Form of LP
  • The standard form of the model, sometimes
    referred to as the canonical form, is written as

(1) (2) (3)
Max Z s.t.
Where all bi are nonnegative.
  • TheoryIf a finite optimal solution exists, there
    is an optimal solution at one of the extreme
    points or vertices of the feasible region.
  • Definition 1 Let P be a polyhedron in
    n-dimensional space, written as . A
    vector x is an extreme point of P if we cannot
    find two vectors y, , both different from
    x, and a scalar such that x ?y (1

(No Transcript)
  • In other words, x is an extreme point if it
    doesn't lie on the line between any two points in
  • Definition 2 Let a be a nonzero row vector in
    n-dimensional space, written as , and
    let b be a scalar.
  • a. The set x ax b is called a
  • b. The set x ax?b is called a half

  • Note that a hyperplane is the boundary of a
    corresponding halfspace. For a polyhedron in
    n-dimensional space, an extreme point is the
    intersection of n non-coplanar hyperplanes. (If
    more than n hyperplanes are intersected, the
    extreme point is degenerated.)
  • For the linear programming problem defined by
    Equations (1) to (3), some of these hyperplanes
    will be of the form 0.

  • The objective must be to maximize.
  • The objective function must be linear in the
    variables and must not contain any constant
  • All variables must be restricted to be
  • Each constraint must be written as a linear
    equation with the variables on the left of the
    equal sign and a positive constant on the right.

  • Minimize ltgt
    Maximize -
  • ltgt
  • ltgt
  • For unrestricted variable ,
  • where 0 and

  • When more than one variable is unrestricted,
  • where and

Example (Transformation)
(No Transcript)
Maximize Subject to
  • When solving two-dimensional problems
    graphically, we saw that solutions were always
    found at extreme points of the feasible region.
    These solutions are called basic solutions.
  • In a problem with n variables and m
    constraints, a basic solution is determined by
    identifying m variables as basic, setting the
    remaining n - m (nonbasic) variables equal to
    zero, and solving the resultant set of
    simultaneous equations.

  • In order for these equations to have a unique
    solution and hence correspond to an extreme
    point, care must be taken in choosing which
    variables to make basic.

Linear Independence
  • Let us consider a system of m linear equations
    in m unknowns, written as Axb.

  • For this system to have a unique solution, the
    matrix A must be invertible or nonsingular that
    is, there must exist another m m matrix B such
    that AB BA I, where I is the m x m identity
    matrix. Such a matrix B is called the inverse of
    A and is unique. It is denoted by .

Definition 3 Let be a
collection of k column vectors, each of dimension
m. We say that these vectors are linearly
independent (LI)if it is not possible to find k
real numbers not all zero such
that , where 0 is
the m-dimensional null vector otherwise, they
are called linearly dependent (LD).
  • Theorem 1 Let A be a square matrix. The
  • statements are equivalent.
  • The determinant of A is nonzero.
  • The matrix A is invertible, as is its transpose
  • The rows and columns of A are linearly
  • For every vector b, the linear system Ax b has
    a unique solution.

Basic Solution
  • Consider a system of equations Ax b, where x
    is an n-dimensional vector, b is an m-dimensional
    vector, and A is an m n matrix. Suppose that
    from the n columns of A we select a set of m
    linearly independent columns (such a set exists
    if the rank of A is m).

  • Definition 4 Given a set of m simultaneous
    linear equations in n unknowns, let B be any
    nonsingular m m matrix made up of columns of A.
    If all the n - m components of x not associated
    with columns of B are set equal to zero, the
    solution to the resulting set of equations is
    said to be a basic solution with respect to the
    basis B.
  • The components of x associated with columns of
    B are called basic variables.

  • Given the assumption that A has full row rank,
    the system Ax b always has a solution and, in
    fact, will always have at least one basic
    solution however, the basic variables in a
    solution are not necessarily all nonzero.
  • Definition 5 A degenerate basic solution is
    said to occur if one or more of the basic
    variables in a basic solution have values of

  • Definition 6 A vector x S x
    Ax b, x?0 is said to be feasible to the linear
    programming problem in standard form a feasible
    solution that is also basic is said to be a basic
    feasible solution (BFS).If this solution is
    degenerate, it is called a degenerate basic
    feasible solution.

  • Theorem 2 Given a linear program in standard
    form, where A is an m n matrix of rank m,
  • a. if there is a feasible solution, there is
    a BFS
  • b. if there is an optimal feasible solution,
    there is an optimal BFS.
  • In general, if there are m constraints and n
    variables, the number of basic solutions will be
    bounded by the number of ways to choose m basic
    variables (or, equivalently, the n -m nonbasic
    variables) from the n variablesi.e.,

(No Transcript)
The upper bound on the number of basic solutions
is computed to be
  • Theorem 3 For any linear program, there is a
    unique extreme point in the feasible region S
    x Ax b, x?0 corresponding to a BFS.
    In addition, there is at least one BFS
    corresponding to each extreme point in S.

  • When more than one BFS maps into a specific
    extreme point, at least one of the binding
    constraints at that extreme point is redundant.
    It could be either one of the structural
    constraints or one of the nonnegativity
  • In either case, the BFSs would all be

  • Definition 7 For a linear program in standard
    form with m constraints, two basic feasible
    solutions are said to be adjacent if they have m
    - 1 basic variables in common.
  • For example, referring to Figure 3.3 we see
    that points 2 and 8 are adjacent. Given that m
    3, these points should have 3-12 basic
    variables in common to be adjacent according to
    Definition 7. (which ones?)

gt If x is better than all its adjacent BFSs,
then x is an optimal BFS. (why?)
gt Convexity of Linear programming.
Simplex Algorithm
  • Step 1 Find an initial BFS to the linear
    program. As the algorithm progresses, the most
    recent BFS will be called the incumbent.
  • Step 2 Determine if the incumbent is optimal. If
    not, move to an adjacent BFS that has a larger z
  • Step 3 Return to Step 2, using the new BFS as
    the incumbent.

  • Although the number of BFSs could be huge, the
    simplex algorithm is seen in practice to be
    highly efficient. Vanderbei 1996 has
    empirically demonstrated that in the absence of
    degeneracy, approximately 0.5(m n) BFSs have to
    be examined before optimality is confirmed (when
    degeneracy is present, this number doubles)

Simplex Tableau
Move all variables to the left hand side of
, and all constants to the right hand side of
(No Transcript)
(No Transcript)
(No Transcript)
The marginal effects of increasing can be
described by the following partial derivatives,

  • In general terms, the equations are written as
  • where Q is the set of nonbasic variables.

What is the Basic Solution?
  • From the general expression of the simplex form
    we read the objective function value and the
    values of the basic variables as

Is the basic solution feasible?
  • In general, a basic solution is feasible if

Is the basic feasible solution optimal?
  • Optimality condition
  • Why?
  • Increasing the value of any nonbasic variable
    (from its current value 0) can not improve the
    current objective function value.

How Can a Nonoptimal Feasible Solution Be
  • The current solution can be improved by
    increasing any nonbasic variable with a negative
    coefficient in Equation E0.
  • In the general case, all nonbasic variables with
    lt 0 are candidates for conversion to basic

How Much Should the Nonbasic Variable Be Changed?
  • When gt 0, a positive value of will
    drive the corresponding basic variable to zero.
    When more than one row has this characteristic,
    the nonbasic variable can increase only to the
    value of the minimum positive ratio if
    feasibility is to be guaranteed. Thus the
    nonbasic variable may increase by

(min. ratio test)
(No Transcript)
How Is a New Simplex Form Obtained?
  • The process of obtaining the new tableau is
    called pivoting or pricing out the entering
    column. It is simply a series of linear
    operations on the equations that results in the
    simplex form of the new set of basic variables.

Suppose the rth row is selected by the min. ratio
test. Then,
Before Pivoting
(No Transcript)
After Pivoting
A Complete Example
(No Transcript)
(No Transcript)
(No Transcript)
Another Example
  • Max Z3x15x2
  • s.t. x1 ?4
  • 2x2 ?12
  • 3x1 2x2 ?18
  • x1 ,x2 ?0

Convert to Canonical Form
  • Max Z3x15x2 0x3 0x40x5
  • s.t. x1 x3 4
  • 2x2 x4 12
  • 3x1 2x2 x5 18
  • x1 ,x2 , x3 ,x4, x5 ?0

Convert to System of Equations
  • Z-3x1-5x2 0x3 0x40x5 0
  • x1 x3 4
  • 2x2 x4 12
  • 3x1 2x2 x5 18
  • Perform elementary row operations (e.r.o.) to
  • keep it in a standard for and try to maximize Z.
  • Maintain implicitly x1 ,x2 , x3 ,x4, x5 ?0

Tableau form
row basic x1 x2 x3 x4 x5 RHS
0 Z -3 -5 0 0 0 0
1 x3 1 0 1 0 0 4
2 x4 0 2 0 1 0 12
3 x5 3 2 0 0 1 18
row basic x1 x2 x3 x4 x5 RHS
0 Z -3 0 0 5/2 0 30
1 x3 1 0 1 0 0 4
2 x2 0 1 0 1/2 0 6
3 x5 3 0 0 -1 1 6
Tableau form
row basic x1 x2 x3 x4 x5 RHS
0 Z 0 0 0 3/2 1 36
1 x3 0 0 1 1/3 -1/3 2
2 x2 0 1 0 1/2 0 6
3 x1 1 0 0 -1/3 1/3 2
  • Tie for the Entering Variable
  • When the steepest ascent rule is used for
    selecting the variable that will enter the basis,
    if there is a tie for the most negative value,
    any one of the tied variables may be chosen

  • Tie for the Leaving Variable
  • The basic variable for any one of the tied
    rows may be chosen arbitrarily to leave the
  • If more than one row has the same positive
    ratio, more than one basic variable will go to
    zero at the next iteration. (degenerate BFS)

  • Definition 8 A basic solution is
    said to be degenerate if the number of structural
    constraints and nonnegativity conditions active
    at x is greater than n.
  • In two dimensions, a degenerate basic solution
    occurs at the intersection of three or more
    lines in three dimensions, a degenerate solution
    occurs at the intersection of four or more
  • The effect of degeneracy is to interrupt the
    steady progress of the simplex algorithm toward

  • When degeneracy occurs, the possibility arises
    that the simplex algorithm will return to a
    previously encountered basis. (Cycling)
  • Nevertheless, cycling can be eliminated by
    modifying the rules used to select the entering
    and leaving variables.

  • The following rules are based on the indices of
    the problem variables (j 1,..., n) and are
    attributable to Bland 1977.
  • Rule for selecting the entering variable From
    all the variables having negative coefficients in
    row 0, select as the entering variable the one
    with the smallest index,
  • Rule for selecting the leaving variable Use the
    standard ratio test to select the leaving
    variable, but if there is a tie in the ratio
    test, select from the tied rows the variable
    having the smallest index.

Alternative optima
  • Although the simplex method automatically stops
    when the first optimal BFS is reached, others can
    be found by forcing a nonbasic variable with zero
    reduced cost to enter the basis.
  • When more than one optimal BFS exists, any
    weighted average of them is also optimal, where
    the weights are nonnegative and sum to 1. Such a
    weighted linear average is called a convex

Unbounded Solution
  • When there is a nonbasic variable with a negative
    coefficient in row 0 and all the other
    coefficients in that column are nonpositive, the
    value of that variable can be increased
    indefinitely without driving any basic variable
    to zero. The objective function can then be made
    arbitrarily large. This condition signals an
    unbounded solution, and there is no optimal
    solution for the model. The algorithm must stop
    if such an event occurs.