- ??????(?)

Chapter 3 Linear Programming Methods

Introduction

- Whereas the simplex method searches for the

optimal solution at the boundary of the feasible

region, the latest developments are based on the

idea of following a path through the interior of

the feasible region until the optimum is reached.

These so-called interior point methods are of

great theoretical importance because they provide

a bound on the computational effort required to

solve a problem that is a polynomial function of

its size. - No polynomial bound is available for the simplex

algorithm. Nevertheless, simplex codes have

proven to be highly efficient in practice and

remain at the center of virtually all commercial

optimization packages.

Standard Form of LP

- The standard form of the model, sometimes

referred to as the canonical form, is written as

(1) (2) (3)

Max Z s.t.

Where all bi are nonnegative.

- TheoryIf a finite optimal solution exists, there

is an optimal solution at one of the extreme

points or vertices of the feasible region. - Definition 1 Let P be a polyhedron in

n-dimensional space, written as . A

vector x is an extreme point of P if we cannot

find two vectors y, , both different from

x, and a scalar such that x ?y (1

-?)z.

(No Transcript)

- In other words, x is an extreme point if it

doesn't lie on the line between any two points in

P. - Definition 2 Let a be a nonzero row vector in

n-dimensional space, written as , and

let b be a scalar. - a. The set x ax b is called a

hyperplane. - b. The set x ax?b is called a half

space.

- Note that a hyperplane is the boundary of a

corresponding halfspace. For a polyhedron in

n-dimensional space, an extreme point is the

intersection of n non-coplanar hyperplanes. (If

more than n hyperplanes are intersected, the

extreme point is degenerated.) - For the linear programming problem defined by

Equations (1) to (3), some of these hyperplanes

will be of the form 0.

PREPARING THE MODEL

- The objective must be to maximize.
- The objective function must be linear in the

variables and must not contain any constant

terms. - All variables must be restricted to be

nonnegative. - Each constraint must be written as a linear

equation with the variables on the left of the

equal sign and a positive constant on the right.

- Minimize ltgt

Maximize - - ltgt
- ltgt
- For unrestricted variable ,
- where 0 and

- When more than one variable is unrestricted,
- where and

Example (Transformation)

Letand

(No Transcript)

Maximize Subject to

GEOMETRIC PROPERTIES OF LPs

- When solving two-dimensional problems

graphically, we saw that solutions were always

found at extreme points of the feasible region.

These solutions are called basic solutions. - In a problem with n variables and m

constraints, a basic solution is determined by

identifying m variables as basic, setting the

remaining n - m (nonbasic) variables equal to

zero, and solving the resultant set of

simultaneous equations.

- In order for these equations to have a unique

solution and hence correspond to an extreme

point, care must be taken in choosing which

variables to make basic.

Linear Independence

- Let us consider a system of m linear equations

in m unknowns, written as Axb.

- For this system to have a unique solution, the

matrix A must be invertible or nonsingular that

is, there must exist another m m matrix B such

that AB BA I, where I is the m x m identity

matrix. Such a matrix B is called the inverse of

A and is unique. It is denoted by .

Definition 3 Let be a

collection of k column vectors, each of dimension

m. We say that these vectors are linearly

independent (LI)if it is not possible to find k

real numbers not all zero such

that , where 0 is

the m-dimensional null vector otherwise, they

are called linearly dependent (LD).

- Theorem 1 Let A be a square matrix. The

following - statements are equivalent.
- The determinant of A is nonzero.
- The matrix A is invertible, as is its transpose

. - The rows and columns of A are linearly

independent. - For every vector b, the linear system Ax b has

a unique solution.

Basic Solution

- Consider a system of equations Ax b, where x

is an n-dimensional vector, b is an m-dimensional

vector, and A is an m n matrix. Suppose that

from the n columns of A we select a set of m

linearly independent columns (such a set exists

if the rank of A is m).

- Definition 4 Given a set of m simultaneous

linear equations in n unknowns, let B be any

nonsingular m m matrix made up of columns of A.

If all the n - m components of x not associated

with columns of B are set equal to zero, the

solution to the resulting set of equations is

said to be a basic solution with respect to the

basis B. - The components of x associated with columns of

B are called basic variables.

- Given the assumption that A has full row rank,

the system Ax b always has a solution and, in

fact, will always have at least one basic

solution however, the basic variables in a

solution are not necessarily all nonzero. - Definition 5 A degenerate basic solution is

said to occur if one or more of the basic

variables in a basic solution have values of

zero.

- Definition 6 A vector x S x

Ax b, x?0 is said to be feasible to the linear

programming problem in standard form a feasible

solution that is also basic is said to be a basic

feasible solution (BFS).If this solution is

degenerate, it is called a degenerate basic

feasible solution.

Foundations

- Theorem 2 Given a linear program in standard

form, where A is an m n matrix of rank m, - a. if there is a feasible solution, there is

a BFS - b. if there is an optimal feasible solution,

there is an optimal BFS. - In general, if there are m constraints and n

variables, the number of basic solutions will be

bounded by the number of ways to choose m basic

variables (or, equivalently, the n -m nonbasic

variables) from the n variablesi.e.,

Example

(No Transcript)

The upper bound on the number of basic solutions

is computed to be

- Theorem 3 For any linear program, there is a

unique extreme point in the feasible region S

x Ax b, x?0 corresponding to a BFS.

In addition, there is at least one BFS

corresponding to each extreme point in S.

- When more than one BFS maps into a specific

extreme point, at least one of the binding

constraints at that extreme point is redundant.

It could be either one of the structural

constraints or one of the nonnegativity

restrictions. - In either case, the BFSs would all be

degenerate.

- Definition 7 For a linear program in standard

form with m constraints, two basic feasible

solutions are said to be adjacent if they have m

- 1 basic variables in common. - For example, referring to Figure 3.3 we see

that points 2 and 8 are adjacent. Given that m

3, these points should have 3-12 basic

variables in common to be adjacent according to

Definition 7. (which ones?)

gt If x is better than all its adjacent BFSs,

then x is an optimal BFS. (why?)

gt Convexity of Linear programming.

Simplex Algorithm

- Step 1 Find an initial BFS to the linear

program. As the algorithm progresses, the most

recent BFS will be called the incumbent. - Step 2 Determine if the incumbent is optimal. If

not, move to an adjacent BFS that has a larger z

value. - Step 3 Return to Step 2, using the new BFS as

the incumbent.

- Although the number of BFSs could be huge, the

simplex algorithm is seen in practice to be

highly efficient. Vanderbei 1996 has

empirically demonstrated that in the absence of

degeneracy, approximately 0.5(m n) BFSs have to

be examined before optimality is confirmed (when

degeneracy is present, this number doubles)

Simplex Tableau

Move all variables to the left hand side of

, and all constants to the right hand side of

.

(No Transcript)

(No Transcript)

(No Transcript)

The marginal effects of increasing can be

described by the following partial derivatives,

- In general terms, the equations are written as
- where Q is the set of nonbasic variables.

What is the Basic Solution?

- From the general expression of the simplex form

we read the objective function value and the

values of the basic variables as

Is the basic solution feasible?

- In general, a basic solution is feasible if

Is the basic feasible solution optimal?

- Optimality condition
- Why?
- Increasing the value of any nonbasic variable

(from its current value 0) can not improve the

current objective function value.

How Can a Nonoptimal Feasible Solution Be

Improved?

- The current solution can be improved by

increasing any nonbasic variable with a negative

coefficient in Equation E0. - In the general case, all nonbasic variables with

lt 0 are candidates for conversion to basic

variables.

How Much Should the Nonbasic Variable Be Changed?

- When gt 0, a positive value of will

drive the corresponding basic variable to zero.

When more than one row has this characteristic,

the nonbasic variable can increase only to the

value of the minimum positive ratio if

feasibility is to be guaranteed. Thus the

nonbasic variable may increase by

(min. ratio test)

(No Transcript)

How Is a New Simplex Form Obtained?

- The process of obtaining the new tableau is

called pivoting or pricing out the entering

column. It is simply a series of linear

operations on the equations that results in the

simplex form of the new set of basic variables.

Pivoting

Suppose the rth row is selected by the min. ratio

test. Then,

Before Pivoting

(No Transcript)

After Pivoting

0.5

0.5

A Complete Example

3

(No Transcript)

(No Transcript)

(No Transcript)

Another Example

- Max Z3x15x2
- s.t. x1 ?4
- 2x2 ?12
- 3x1 2x2 ?18
- x1 ,x2 ?0

Convert to Canonical Form

- Max Z3x15x2 0x3 0x40x5
- s.t. x1 x3 4
- 2x2 x4 12
- 3x1 2x2 x5 18
- x1 ,x2 , x3 ,x4, x5 ?0

Convert to System of Equations

- Z-3x1-5x2 0x3 0x40x5 0
- x1 x3 4
- 2x2 x4 12
- 3x1 2x2 x5 18
- Perform elementary row operations (e.r.o.) to
- keep it in a standard for and try to maximize Z.

- Maintain implicitly x1 ,x2 , x3 ,x4, x5 ?0

Tableau form

row basic x1 x2 x3 x4 x5 RHS

0 Z -3 -5 0 0 0 0

1 x3 1 0 1 0 0 4

2 x4 0 2 0 1 0 12

3 x5 3 2 0 0 1 18

row basic x1 x2 x3 x4 x5 RHS

0 Z -3 0 0 5/2 0 30

1 x3 1 0 1 0 0 4

2 x2 0 1 0 1/2 0 6

3 x5 3 0 0 -1 1 6

Tableau form

row basic x1 x2 x3 x4 x5 RHS

0 Z 0 0 0 3/2 1 36

1 x3 0 0 1 1/3 -1/3 2

2 x2 0 1 0 1/2 0 6

3 x1 1 0 0 -1/3 1/3 2

SPECIAL SITUATIONS

- Tie for the Entering Variable
- When the steepest ascent rule is used for

selecting the variable that will enter the basis,

if there is a tie for the most negative value,

any one of the tied variables may be chosen

arbitrarily.

- Tie for the Leaving Variable
- The basic variable for any one of the tied

rows may be chosen arbitrarily to leave the

basis. - If more than one row has the same positive

ratio, more than one basic variable will go to

zero at the next iteration. (degenerate BFS)

- Definition 8 A basic solution is

said to be degenerate if the number of structural

constraints and nonnegativity conditions active

at x is greater than n. - In two dimensions, a degenerate basic solution

occurs at the intersection of three or more

lines in three dimensions, a degenerate solution

occurs at the intersection of four or more

planes. - The effect of degeneracy is to interrupt the

steady progress of the simplex algorithm toward

optimality.

Example

- When degeneracy occurs, the possibility arises

that the simplex algorithm will return to a

previously encountered basis. (Cycling) - Nevertheless, cycling can be eliminated by

modifying the rules used to select the entering

and leaving variables.

- The following rules are based on the indices of

the problem variables (j 1,..., n) and are

attributable to Bland 1977. - Rule for selecting the entering variable From

all the variables having negative coefficients in

row 0, select as the entering variable the one

with the smallest index, - Rule for selecting the leaving variable Use the

standard ratio test to select the leaving

variable, but if there is a tie in the ratio

test, select from the tied rows the variable

having the smallest index.

Alternative optima

- Although the simplex method automatically stops

when the first optimal BFS is reached, others can

be found by forcing a nonbasic variable with zero

reduced cost to enter the basis. - When more than one optimal BFS exists, any

weighted average of them is also optimal, where

the weights are nonnegative and sum to 1. Such a

weighted linear average is called a convex

combination.

Unbounded Solution

- When there is a nonbasic variable with a negative

coefficient in row 0 and all the other

coefficients in that column are nonpositive, the

value of that variable can be increased

indefinitely without driving any basic variable

to zero. The objective function can then be made

arbitrarily large. This condition signals an

unbounded solution, and there is no optimal

solution for the model. The algorithm must stop

if such an event occurs.

Example