Title: UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2006
1 UMass Lowell Computer Science 91.503 Analysis
of Algorithms Prof. Karen Daniels Fall, 2006
- Lecture 9
- Wednesday, 11/15/06
- Linear Programming
2Overview
- Motivation Basics
- Standard Slack Forms
- Formulating Problems as Linear Programs
- Simplex Algorithm
- Example
- High-Level Algorithm
- Correctness
- Roadmap
- Key Concepts
- Initial Basic Feasible Solution
- Duality
- Literature Case Study
3Motivation Basics
4Motivation A Political Problem
Goal Win election by winning majority of votes
in each region.
100,000 voters
200,000 voters
50,000 voters
Thousands of voters who could be won with 1,000
of ads
Subgoal Win majority of votes in each region
while minimizing advertising cost.
source 91.503 textbook Cormen et al.
5Motivation A Political Problem (continued)
Thousands of voters representing majority.
urban
suburban
rural
source 91.503 textbook Cormen et al.
6General Linear Programs
real numbers
Linear function
variables
Linear constraints
Linear inequalities
source 91.503 textbook Cormen et al.
7Overview of Linear Programming
Objective function
Convex feasible region
Objective value
source 91.503 textbook Cormen et al.
8Standard Slack Forms
9Standard Form
objective function
constraints
source 91.503 textbook Cormen et al.
10Standard Form (compact)
n-dimensional vectors
m-dimensional vector
mxn matrix
Can specify linear program in standard form by
(A,b,c).
source 91.503 textbook Cormen et al.
11Converting to Standard Form
source 91.503 textbook Cormen et al.
12Converting to Standard Form (continued)
Transforming minimization to maximization
Negate coefficients
source 91.503 textbook Cormen et al.
13Converting to Standard Form (continued)
Giving each variable a non-negativity constraint
New non-negativity constraints
If xj has no non-negativity constraint, replace
each occurrence of xj with xj xj.
source 91.503 textbook Cormen et al.
14Converting to Standard Form (continued)
Transforming equality constraints to inequality
constraints
source 91.503 textbook Cormen et al.
15Converting to Standard Form (continued)
Changing sense of an inequality constraint
Rationale
source 91.503 textbook Cormen et al.
16Converting Linear Programs into Slack Form
for algorithmic ease, transform all constraints
except non-negativity ones into equalities
slack variable
for inequality constraint
define slack
instead of s
non-basic variables
basic variables
source 91.503 textbook Cormen et al.
17Converting Linear Programs into Slack
Form (continued)
objective function
source 91.503 textbook Cormen et al.
18Converting Linear Programs into Slack
Form (continued)
set of indices of basic variables
set of indices of non-basic variables
Compact Form (N, B, A, b, c, v)
Slack Form Example
Compact Form
negative of slack form coefficients
source 91.503 textbook Cormen et al.
19 Formulating Problems as Linear Programs
20Shortest Paths
Single-pair shortest path minimize distance
from source s to sink t.
Can we replace maximize with minimize here? Why
or why not?
source 91.503 textbook Cormen et al.
21Maximum Flow
source 91.503 textbook Cormen et al.
22Minimum Cost Flow
source 91.503 textbook Cormen et al.
23Multicommodity Flow
should be si
source 91.503 textbook Cormen et al.
24Simplex Method
25Solving a Linear Program
- Simplex algorithm
- Geometric interpretation
- Visit vertices on the boundary of the simplex
representing the convex feasible region - Transforms set of inequalities using process
similar to Gaussian elimination - Run-time
- not polynomial in worst-case
- often very fast in practice
- Ellipsoid method
- Run-time
- polynomial
- slow in practice
- Interior-Point methods
- Run-time
- polynomial
- for large inputs, performance can be competitive
with simplex method - Moves through interior of feasible region
source 91.503 textbook Cormen et al.
26Simplex Algorithm ExampleBasic Solution
Standard Form
Slack Form
Basic Solution
Basic Solution set each nonbasic variable to 0.
source 91.503 textbook Cormen et al.
27Simplex Algorithm Example Reformulating the LP
Model
Main Idea In each iteration, reformulate the LP
model so basic solution has larger objective value
Select a nonbasic variable whose objective
coefficient is positive x1
Increase its value as much as possible.
Identify tightest constraint on increase.
For basic variable x6 of that constraint, swap
role with x1.
Rewrite other equations with x6 on RHS.
new objective value
entering variable
PIVOT
source 91.503 textbook Cormen et al.
leaving variable
28Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x3 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
29Simplex Algorithm Example Reformulating the LP
Model
Next Iteration select x2 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution
source 91.503 textbook Cormen et al.
30Simplex Algorithm Pivoting
entering variable
leaving variable
Rewrite the equation that has xl on LHS to have
xe on LHS
Update remaining equations by substituting RHS of
new equation for each occurrence of xe.
Do the same for objective function.
Update sets of nonbasic, basic variables.
source 91.503 textbook Cormen et al.
31Simplex Algorithm Pseudocode
initial basic solution
to be defined later (detects infeasibility)
detects unboundedness
optimal solution
source 91.503 textbook Cormen et al.
32Correctness Roadmap (Key
Pieces)
Lemma 29.3 Algebraic lemma
Lemma 29.8 Weak LP duality
Lemma 29.4 Slack form uniqueness
Corollary 29.9 Conditions for which feasible
solutions for primal, dual programs are optimal
Lemma 29.1 Pivot results
Lemma 29.6 Tie-breaking
Lemma 29.11 Laux
Lemma 29.2 Basic solution feasible -gt if SIMPLEX
finds solution it is feasible if reports
unbounded, then model is unbounded
Lemma 29.12 Infeasibility detection
Theorem 29.10 LP duality SIMPLEX primal result
is optimal dual is optimal
Theorem 29.13 Fundamental Theorem of Linear
Programming For LP model in standard form,
either 1. exists optimal solution with finite
objective function value SIMPLEX returns one,
or 2. infeasible SIMPLEX returns INFEASIBLE,
or 3. Unbounded SIMPLEX returns UNBOUNDED
33Proof of Correctness Key Concepts
- Initial Basic Feasible Solution
34Finding an Initial Solution
An LP model whose initial basic solution is not
feasible
source 91.503 textbook Cormen et al.
35Finding an Initial Solution(continued)
Auxiliary LP model Laux
source 91.503 textbook Cormen et al.
36Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
37Finding an Initial Solution(continued)
Original LP model
Laux
Laux in slack form
source 91.503 textbook Cormen et al.
38Finding an Initial Solution(continued)
PIVOT
PIVOT
source 91.503 textbook Cormen et al.
39Finding an Initial Solution(continued)
source 91.503 textbook Cormen et al.
40Proof of Correctness Key Concepts
41Linear Programming Duality
max becomes min
x variables go away
RHS coefficients swap places with objective
function coefficients
y variables appear
sense changes
source 91.503 textbook Cormen et al.
42Duality Example
source 91.503 textbook Cormen et al.
43Weak Linear Programming Duality
Any feasible solution to primal LP has value no
greater than that of any feasible solution to the
dual LP.
source 91.503 textbook Cormen et al.
44Weak Linear Programming Duality (continued)
source 91.503 textbook Cormen et al.
45Finding a Dual Solution
Finding a dual solution whose value is equal to
that of an optimal primal solution
source 91.503 textbook Cormen et al.
46Optimality
source 91.503 textbook Cormen et al.
47Literature Case Study