The simplex algorithm

1
The simplex algorithm

• Section 29.3
• Presented By Doug Nickerson
• 11/20/07

2
The simplex algorithm

• Classical method for solving linear programs
  (LPs)
• Worst case running time is not polynomial
• However, its quite fast in practice (most cases)
• Provides insight into linear programs via its
  geometrical interpretation as well as similarity
  to Gaussian elimination
• Could be called Gaussian elimination for
  inequalities

3
The simplex algorithm

• Iterates through several equivalent slack forms
  of the linear program
• Each iteration- increases the objective value
  by at least 0- selects one non-basic and one
  basic variable to swap- rewrites the LP in a new
  (but equivalent) slack form which has the 2
  variables swapped
• A basic solution- results from setting all
  non-basic variables to value 0- can be easily
  obtained from the slack form- always corresponds
  to a vertex of the simplex
• Hence the simplex algorithm is simply walking
  from vertex to vertex on the simplex

4
An example of the simplex algorithm

• Lets run the simplex algorithm on an example
  LP
• Any setting of the non-basic variables (x1, x2,
  x3), will yield and values for the basic
  variables (x4, x5, x6)
• The basic solution here is (x1, x2, , x6)
  (0,0,0,30,24,36) and objective value z
  3010200

Standard form
Slack form
5
An example of the simplex algorithm

• Goal is to rewrite a new slack form which has a
  basic solution that increases the objective value
  z.
• Note, a basic solution is a feasible basic
  solution if the basic variables satisfy the
  non-negativity requirements.
• Choose a non-basic variable that has a positive
  coefficient in the obj. value expression say x1
  in this case
• Increasing x1 increases z, but one of the
  constraints will limit x1 more than the others
  in this case, the last one

6
An example of the simplex algorithm

• Now, weve determined the 2 variables to swap x1
  will become a basic variable and x6 will become
  non-basic.
• Solve the most restrictive constraint for x1
• Then substitute that into the other constraints
  and the objective value equation to get the new
  slack form

7
An example of the simplex algorithm

• Simplify and we have the result of the first
  iteration
• Note the constant in the objective value
  expression 27-This is the new objective value
  for the new basic solution (x1, x2, , x6)
  (9,0,0,21,6,0)
• This procedure is called pivoting and is at the
  heart of the simplex algorithm

Initial slack form
After x1 and x6 were swapped
8
An example of the simplex algorithm

• Now we repeat- select a non-basic variable with
  a positive coefficient in the z-equation we
  call it the entering variable- determine which
  constraint is the tightest and call the
  basic variable of that constraint the leaving
  variable- perform the pivot to get the new
  slack form
• New basic solution (33/4, 0, 3/2, 69/4, 0, 0)
  gives new objective value z 111/4 27.75 gt 27

9
An example of the simplex algorithm

• Finally, we only have one choice of the entering
  variable
• Final slack form
• New basic solution (8, 4, 0, 18, 0, 0) gives new
  objective value z 28 gt 27.75

10
Pivoting

• Fundamental procedure of simplex algorithm
• Input (N, B, A, b, c, v, l, e)- N and B are
  lists of the non-basic and basic variables- A is
  the matrix of constraint coefficients- b is the
  vector of constants in the constraints- c is the
  vector of coefficients in the objective
  equation- v is the constant in the objective
  equation- l and e are the indices of the leaving
  and entering vars.

11
Pivoting

• Output (N, B, A, b, c, v) which describes
  the new slack form having variables xe and xl
  exchanged.
• Pseudo-code part 1 (page 795)
• These lines determine the coefficients of the new
  constraint with xe as the basic variable.
• It essentially solves the equation for xe

Ex N1,2,3, e1, l6) x636 4 x1 x2
2 x3 b6 a61x1 a62x2 a63x3 x1
b6/a61 a62/a61 x2 a63/a61 x3
x6/a61 b1 a12x2 a13x3 a16x6
PIVOT(N, B, A, b, c, v, l, e) be ? bl / ale
for each j N e do aej ? alj /
ale ael ? 1 / ale
12
Pivoting

• Pseudo-code part 2
• These lines determine the coefficients of the
  remaining constraints (the ones where the basic
  variable remains).
• It performs the substitution of xe and resulting
  simplification (collecting coefficients).

Ex N1,2,3, B4,5,6, e1, l6) x430
x1 x2 3 x3 b4 a41x1 a42x2
a43x3 bi aie x1 aij x2 aij
x3 x1be aejx2 aejx3 aelx6 x4 bi
aie () aij x2 aij x3 (bi- aiebe)
(aij-aie aej)x2 (aij-aie aej)x3
aieaelx6
for each i B l do bi ? bi aie
be for each j N e do
aij ? aij aie aej ail ? - aie ael
13
Pivoting

• Pseudo-code part 3
• The first 4 lines determine the coefficients of
  the objective expression by doing the
  substitution of xe and resulting simplification
  (collecting coefficients).
• The last 3 lines simply swap the e and l indices
  in the sets N and B and return the new slack form.

Ex N1,2,3, B4,5,6, e1, l6) case i
4 z 0 3 x1 x2 2 x3 v c1x1
c2x2 c3 x3 v ce() cjx2 cj
x3 x1be aejx2 aejx3 aelx6 z (v - ce
be) (cj- ce aej)x2 (cj- ce aej)x3
ce aelx6
v ? v ce be for each j N e do cj
? cj ce aej c l ? - ce ael N(N-e) U
l B(B-l) U e return (N, B, A, b, c,
v)
14
Pivoting

• Time complexity of PIVOT- Contains 2 loops over
  j and a double nested loop over i and j plus
  some other O(1) operations- Each j-loop iterates
  over N-e which has N-1n-1- The i-loop
  iterates over B-l which has B-1m-1-
  Therefore T(n,m) 2(n-1) (n-1)(m-1) O(1)
  nm n m
  O(1)- Clearly nm dominates, hence O(nm)
• Lemma 29.1

15
The formal simplex algorithm

• Intro - questions

16
The formal simplex algorithm

• Pseudo-code part 1 (page 797)

17
The formal simplex algorithm

• Pseudo-code part 2
• Lemma 29.2

18
The formal simplex algorithm - Proofs

• Just used linear programming to solve problem
  with an efficient algorithm
• Linear programming is slower in theory and in
  practice in most cases
• Need to recognize when new problems can be solved
  via linear programming

19
The formal simplex algorithm - Proofs

• Just used linear programming to solve problem
  with an efficient algorithm
• Linear programming is slower in theory and in
  practice in most cases
• Need to recognize when new problems can be solved
  via linear programming

20
The formal simplex algorithm - Proofs

• Just used linear programming to solve problem
  with an efficient algorithm
• Linear programming is slower in theory and in
  practice in most cases
• Need to recognize when new problems can be solved
  via linear programming

21
Termination

• Just used linear programming to solve problem
  with an efficient algorithm
• Linear programming is slower in theory and in
  practice in most cases
• Need to recognize when new problems can be solved
  via linear programming

22
Termination

• Just used linear programming to solve problem
  with an efficient algorithm
• Linear programming is slower in theory and in
  practice in most cases
• Need to recognize when new problems can be solved
  via linear programming