UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2004

1
UMass Lowell Computer Science 91.503 Analysis
of Algorithms Prof. Karen Daniels Fall, 2004

• Lecture 9
• Wednesday, 11/17/04
• Linear Programming

2
Overview

• Motivation Basics
• Standard Slack Forms
• Formulating Problems as Linear Programs
• Simplex Algorithm
• Duality
• Initial Basic Feasible Solution
• Literature Case Study

3
Motivation Basics
4
Motivation A Political Problem
Goal Win election by winning majority of votes
in each region.
100,000 voters
200,000 voters
50,000 voters
Thousands of voters who could be won with 1,000
of ads
Subgoal Win majority of votes in each region
while minimizing advertising cost.
source 91.503 textbook Cormen et al.
5
Motivation A Political Problem (continued)
Thousands of voters representing majority.
urban
suburban
rural
source 91.503 textbook Cormen et al.
6
General Linear Programs
real numbers
Linear function
variables
Linear constraints
Linear inequalities
source 91.503 textbook Cormen et al.
7
Overview of Linear Programming
Objective function
Convex feasible region
Objective value
source 91.503 textbook Cormen et al.
8
Standard Slack Forms
9
Standard Form
objective function
constraints
source 91.503 textbook Cormen et al.
10
Standard Form (compact)
n-dimensional vectors
m-dimensional vector
mxn matrix
Can specify linear program in standard form by
(A,b,c).
source 91.503 textbook Cormen et al.
11
Converting to Standard Form
source 91.503 textbook Cormen et al.
12
Converting to Standard Form (continued)
Transforming minimization to maximization
Negate coefficients
source 91.503 textbook Cormen et al.
13
Converting to Standard Form (continued)
Giving each variable a non-negativity constraint
New non-negativity constraints
If xj has no non-negativity constraint, replace
each occurrence of xj with xj xj.
source 91.503 textbook Cormen et al.
14
Converting to Standard Form (continued)
Transforming equality constraints to inequality
constraints
source 91.503 textbook Cormen et al.
15
Converting to Standard Form (continued)
Changing sense of an inequality constraint
Rationale
source 91.503 textbook Cormen et al.
16
Converting Linear Programs into Slack Form
for algorithmic ease, transform all constraints
except non-negativity ones into equalities
slack variable
for inequality constraint
define slack
instead of s
non-basic variables
basic variables
source 91.503 textbook Cormen et al.
17
Converting Linear Programs into Slack
Form (continued)
objective function
source 91.503 textbook Cormen et al.
18
Converting Linear Programs into Slack
Form (continued)
set of indices of basic variables
set of indices of non-basic variables
Compact Form (N, B, A, b, c, v)
Slack Form Example
Compact Form
negative of slack form coefficients
source 91.503 textbook Cormen et al.
19
Formulating Problems as Linear Programs
20
Shortest Paths
Single-pair shortest path minimize distance
from source s to sink t.
Can we replace maximize with minimize here? Why
or why not?
source 91.503 textbook Cormen et al.
21
Maximum Flow
source 91.503 textbook Cormen et al.
22
Minimum Flow
source 91.503 textbook Cormen et al.
23
Multicommodity Flow
should be si
source 91.503 textbook Cormen et al.
24
Simplex Method
25
Solving a Linear Program

• Simplex algorithm
• Geometric interpretation
• Visit vertices on the boundary of the simplex
  representing the convex feasible region
• Transforms set of inequalities using process
  similar to Gaussian elimination
• Run-time
• not polynomial in worst-case
• often very fast in practice
• Ellipsoid method
• Run-time
• polynomial
• slow in practice
• Interior-Point methods
• Run-time
• polynomial
• for large inputs, performance can be competitive
  with simplex method
• Moves through interior of feasible region

source 91.503 textbook Cormen et al.
26
Simplex Algorithm ExampleBasic Solution
Standard Form
Slack Form
Basic Solution
Basic Solution set each nonbasic variable to 0.
source 91.503 textbook Cormen et al.
27
Simplex Algorithm Example Reformulating the LP
Model
Main Idea In each iteration, reformulate the LP
model so basic solution has larger objective value
Select a nonbasic variable whose objective
coefficient is positive x1
Increase its value as much as possible.
Identify tightest constraint on increase.
For basic variable x6 of that constraint, swap
role with x1.
Rewrite other equations with x6 on RHS.
new objective value
entering variable
PIVOT
source 91.503 textbook Cormen et al.
leaving variable
28
Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x3 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
29
Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x2 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
30
Simplex Algorithm Issues to be Addressed
source 91.503 textbook Cormen et al.
31
Simplex Algorithm Pivoting
entering variable
leaving variable
Rewrite the equation that has xl on LHS to have
xe on LHS
Update remaining equations by substituting RHS of
new equation for each occurrence of xe.
Do the same for objective function.
Update sets of nonbasic, basic variables.
source 91.503 textbook Cormen et al.
32
Simplex Algorithm Pivoting (continued)
source 91.503 textbook Cormen et al.
33
Simplex Algorithm Pseudocode
Feasibility initial basic solution
to be defined later (detects infeasibility)
optimal solution
source 91.503 textbook Cormen et al.
34
Simplex Algorithm Correctness Proof Strategy

• If SIMPLEX has initial feasible solution and
  eventually terminates, then it either
• returns a feasible solution or
• determines that the LP is unbounded.
• SIMPLEX terminates.
• SIMPLEXs solution is optimal.

source 91.503 textbook Cormen et al.
35
Simplex Algorithm Correctness Proof Approach,
Part I

• If SIMPLEX has initial feasible solution and
  eventually terminates, then it either
• returns a feasible solution or
• determines that the LP is unbounded.

source 91.503 textbook Cormen et al.
36
Simplex Algorithm Correctness Proof Approach,
Part II
source 91.503 textbook Cormen et al.
37
Duality

• Simplex Algorithm Correctness Proof Approach,
  Part III

38
Linear Programming Duality
max becomes min
x variables go away
RHS coefficients swap places with objective
function coefficients
y variables appear
sense changes
source 91.503 textbook Cormen et al.
39
Duality Example
source 91.503 textbook Cormen et al.
40
Weak Linear Programming Duality
Any feasible solution to primal LP has value no
greater than that of any feasible solution to the
dual LP.
source 91.503 textbook Cormen et al.
41
Weak Linear Programming Duality (continued)
source 91.503 textbook Cormen et al.
42
Finding a Dual Solution
Finding a dual solution whose value is equal to
that of an optimal primal solution
source 91.503 textbook Cormen et al.
43
Optimality
source 91.503 textbook Cormen et al.
44
Initial Basic Feasible Solution
45
Finding an Initial Solution
An LP model whose initial basic solution is not
feasible
source 91.503 textbook Cormen et al.
46
Finding an Initial Solution(continued)
Auxiliary LP model Laux
source 91.503 textbook Cormen et al.
47
Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
48
Finding an Initial Solution(continued)
Original LP model
Laux
Laux in slack form
source 91.503 textbook Cormen et al.
49
Finding an Initial Solution(continued)
PIVOT
PIVOT
source 91.503 textbook Cormen et al.
50
Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
51
Fundamental Theorem of Linear Programming
source 91.503 textbook Cormen et al.