Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables

1
Easy Optimization Problems, Relaxation,Local
Processingfor a small subset of variables
2
Different types of relaxation

• Variable by variable relaxation strict
  minimization
• Changing a small subset of variables
  simultaneously Window strict minimization
  relaxation
• Stochastic relaxation may increase the energy
  should be followed by strict minimization

3
Easy to solve problems

• Quadratic functional with / without
  linear equality constraints
• Solve a linear system of equations
• Quadratization of the functional P1, Pgt2
• Linearization of the constraints P2
• Inequality constraints active set method
• Linear functional and linear constraints
• Linearization of the quadratic functional

4
Linear programming

• minimize/maximize a linear function
• under equality/inequality linear constraints
• Standard form
• The region satisfying all the constraints is the
  feasible region and it is convex

5
The basic mechanism of the simplex method A
simple example
6
(No Transcript)
7
(No Transcript)
8
Linear programming (cont.)

• The number of corner points is finite
• The global maximum is at the corner point in
  which Z(x) is greater or equal to the value of Z
  at all adjacent corner points
• The simplex method (Dantzig 1948) starts at a
  feasible corner point and visited a sequence of
  corner points until a maximum is obtained
• of iterations is almost always O(M) or O(N)
  whichever is larger, but can become exponential
  for pathological cases

9
The basic mechanism of the simplex method A
simple example

• Standard form without equality constraints
• Start at the origin always a feasible corner
  point
• N2 , M3 at most 10 corner points, but only 5
  are feasible
• at (x1,x2)(0,0) the two last constraints
  intersect

10
(No Transcript)
11
The basic mechanism of the simplex method A
simple example

• Add slack variables which transform inequality
• constraints to equality constraints
• Start at the origin (x1,x2,s1,s2,s3)(0,0,2,3,4)
  , Z0
• move to another corner point by letting, say, x1
  grow
• x1 may grow until it hits another corner point,
  in which a different constraint holds gt setting
  some si0

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
The basic mechanism of the simplex method A
simple example

• Divide all variables into two groups
  basic/nonbasic
• At the origin (x1,x2,s1,s2,s3)(0,0,2,3,4)
• Choose a nonbasic that maximizes Z x1 , this is
  the entering basic variable
• x1 is increased until it hits the constraint
• There, x12 gt s10 and this is the
  leaving basic variable
• At (2,0) (x1,x2,s1,s2,s3)(2,0,0,3,4), Z30
• Update the equations and continue until no
  further increase in Z is available
• Automatic exchange of variables Simplex Tableau

16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Simplex Tableau with inequality constraints

• In proper form
• Exactly 1 basic variable per equation
• The coefficient of each basic variable is 1 and
  this is the only non zero entry in its column
• The RHS reveal the values of all basic variables
• The entering basic variable has the most negative
  entry in the 0th row (for the objective Z)
• The leaving basic variable is the one that
  minimizes RHS/coefficient of entering variable
• Set the pivot to 1 and use it to eliminate all
  other non zeros in its column
• The maximum is achieved when the 0th row 0

20
basic
nonbasic 0
Slack variables
s1 s2 s3
s1 s2 s3
21
Minimum Ratio Test
entering
basic
leaving
basic
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Simplex Tableau with inequality (less than),
equality and greater than constraints

• If an equality constraints are involved, e.g.,
  x1x24
• The origin is not feasible
• Add an artificial variable to each equality
  constraint
• x1x2t14
• If a constraint is with greater than sign
  3x12x2 16
• The origin is not feasible
• Add a slack variable and an artificial variable
  to each greater than constraint
  3x12x2-s1t116
• In order to find a starting feasible corner point
  for the original LP, solve a Phase 1 LP in which
  the objective is to minimize the sum of all
  artificial variables
• minimize Sti until all ti0 gt feasible for
  the original LP

26
Simplex Tableau in general

• A general LP problem involves
• N original variables
• L less than constraints
• E equality constraints
• G greater than constraints
• Add LG slack variables
• Add EG artificial variables
• To find a starting feasible corner point for LP,
  solve a Phase 1 LP minimize Sti
  (sum of artificial variables)
• INFEASIBILE if at the end of Phase 1 Stigt0
• If Sti0 continue to solve the original LP
• UNBOUNDED an entering basic variable is unlimited

27
Linear programming

• The Simplex method small and large problems
• Interior point methods very large problems
• (Karmarker 1984, polynomial-time algorithm)
• Within ML should not exceed 100 variables
• Many available software MATLAB, numerical
  recipes,
• Adjust your problem to the used software
• Linearization of both the energy functional and
  the constraints the placement problem under pair
  wise non-overlap constraints

28
(No Transcript)
29
Exc6 Window relaxation for the graph drawing
problem

• Consider the following window W of 3x3 squares
  containing the nodes m,n and p
• m is of size 1x1 located at (2,2)
• n is of size 0.8x0.8 located at (3.4,3.2)
• p is of size 0.5x0.5 located at (2.5,3).
• Find a correction to the locations of m,n and p
  such that the quadratic energy is minimized
  subject to inequality constraint demands that the
  area of nodes at each square lt 0.3

(4,4)
(5,4)
8
4
9
7
1
n
p
4
3
(0.5,1.5)
2
5
6
m
1
3
2
(1,1)
30

1. Calculate the current amount of nodes area
   present in each of the 9 squares
2. Calculate amkx e the change (per unit length) in
   the amount of nodes area induced by a small
   change in the x direction of node m to square k,
   k1,,9. Similarly calculate amky , ankx , anky ,
   apkx and apky
3. Write the quadratic energy E as a function of the
   corrections to the variables in W
4. Calculate the current value of E
5. Write the 9 inequalities constraints associated
   with each square
6. Choose the active set of constraints and write
   the Lagrangian
7. Calculate the resulting system of equations and
   solve it
8. Does the solution seem to be reasonable?
9. Choose .25 of the solution, does E decrease at
   that point?
10. Write the linear programming formulation

31
Exc6 Window relaxation for the graph drawing
problem

• Given a graph which is initially drawn at
• Introduce a grid of mxm squares, each square of
  size hx by hy
• Pick a window W of squares
• Define by akix (akiy) the change in the total
  area in the kth square per small change in
• 1. How should akix (akiy) be calculated
• 2. Write the quadratic energy minimization
  problem under equidensity constraints in W
• 3. Write the resulting linear system of equations
• 4. Write a linear programming formulation