ESI 4313 Operations Research 2

1
ESI 4313Operations Research 2

• Nonlinear Programming Model
• (Lecture sets 45)

2
Differences betweenLP and NLP

• The feasible region of an LP is always a convex
  set.
• in fact, a polyhedron
• Isocost/profit curves are straight lines (or
  planes)
• an LP always has an extreme point optimal
  solution (if an optimal solution exists)

3
Finding the optimal solution for an LP
x2
20x110x2160
10
x16
z40000
(4½,7)
z20000
10x115x2150
5
z50500
x1
8
6
4
4
Differences betweenLP and NLP

• These results do not generalize to NLP
• Consider the following example
• A company can produce a good using capital and
  labor.
• K units of capital and L units of labor yields KL
  units of the good
• Capital can be purchased at 200/unit, while
  labor can be purchased at 1/unit
• A total budget of 40 is available

5
Example 1

• NLP model
• maximize z KL
• subject to
• 200K L ? 40
• K, L ? 0

6
Example 1

• Ignoring the budget constraint, we can draw the
  isoprofit lines for the objective function
• Rewrite z KL as L z/K
• Plot this function for different values of z

7
Isoprofit Lines
8
Example 1

• Now also plot the feasible region of the NLP
  problem

L 40 200 K
9
Differences betweenLP and NLP

• We see that for an NLP the optimal solution is
  not necessarily an extreme point of the feasible
  region even if the feasible region is a
  polyhedron
• But in the example the optimal solution is on
  the boundary of the feasible region
• Is this true in general?

10
Example 2

• Consider a simplified version of the location
  problem with Euclidean distances
• In particular, let there be only 1 customer
  location, (2,2)
• But suppose that the facility location is
  constrained to be in a particular area

11
Example 2a

• Suppose that we require that the facility is in
  the square 0,1?0,1, i.e., we have the
  constraints
• so the full problem is

12
Example 2a

• Graphically

? y
(2,2)
x ?
13
Example 2a

• Again, the optimal solution is on the boundary of
  the feasible region
• Again, is this true in general?

14
Example 2b

• Suppose that we require that the facility is in
  the square 0,3?0,3, i.e., we have the
  constraints
• so the full problem is

15
Example 2b

• Graphically

? y
(2,2)
x ?
16
Example 2b

• In this case, the optimal solution is in the
  interior of the feasible region
• Conclusion
• For general NLP problems, we have lost the
  structure of optimal solutions that we found for
  LP problems
• We need different approaches, concepts, etc. to
  help us solve such problems

17
Example 3

• We could have found an even simpler example
• minimize f(x) x2
• subject to
• -1 ? x ? 1

18
One-dimensional NLP problems

• A problem with only a single decision variable is
  called a one-dimensional optimization problem.
• If we restrict ourselves to LP, one-dimensional
  problems are trivially solvable. (why? how?)
• But even one-dimensional NLP problems can be
  challenging!

19
Example 4

• minimize f(x)
• subject to
• 0 ? x ? 360

20
Local Optima

• For any NLP problem, a feasible vector x is a
• local maximum if f(x) ? f(x) for all (feasible)
  x that are close to x.
• local minimum if f(x) ? f(x) for all (feasible)
  x that are close to x.

21
Local Optima

• What does for all x that are close to x mean?
• For all x such that

22
Example 4
examples of local maximum
examples of local minimum
23
Local Optima

• Many algorithms exist for finding a local optimum
  of an NLP
• some of these we will study in this class
• Solvers like LINGO and Excel Solver only find a
  local optimum !!
• if you want to find the global optimum, you need
  to do more work

24
Example 2

• LINGO solution

Local optimal solution found at iteration
22 Objective value
5058.530
Variable Value Reduced Cost
X 3.200526
0.000000 Y
6.220236 0.000000
Row Slack or Surplus Dual
Price 1
5058.530 -1.000000
25
Local Optima

• For an LP problem, any local optimum is a global
  optimum
• why?
• Can we find types of NLP problems for which this
  also holds?
• this would ensure that LINGO or Excel finds the
  global optimum of the NLP

26
Convex and concave functions

• A very important class of nonlinear functions are
  convex and concave functions
• Let f(x1,,xn) be a function defined for all
  vectors x(x1,,xn) in some convex set S

27
Convex sets

• Examples of sets

convex
non-convex
28
Convex sets

• Recall that a set is convex if the line segment
  joining any pair of points in the set is
  completely contained in the set
• More formally

29
Convex functions

• The function f is a convex function if

30
Example convex function

• f(x) x2

?x(1- ?)x
x
x
31
Concave functions

• The function f is a concave function if

32
Example concave function

• f(x) -x2

x
x
?x(1- ?)x
33
Examplenonconvex/nonconcave function
x
x
34
Importance of concave functions in NLP

• Suppose that we have an NLP with the following
  properties
• the feasible region, say S, is convex
• the objective function, say f, is concave
• the objective is to maximize the value of the
  objective function

35
Importance of concave functions in NLP

• Then
• Any local maximum is a global maximum!
• How can we prove this mathematically?
• using a proof by contradiction
• we assume that the result is false, and then
  derive a contradiction, i.e., something that is
  clearly not true
• this then implies that the result must be true

36
Importance of concave functions in NLP

• Suppose there exists a solution x that is a
  local maximum, but not a global maximum
• Since x is not a global maximum, there exists a
  solution x with the property that f(x) gt f(x)
• Now use the fact that f is concave

37
Importance of concave functions in NLP

• Since f is concave, we have that

38
Importance of concave functions in NLP

• If ? is very close to 1, then
• ?x(1-?)x is very close to x
• f (?x(1-?)x ) gt f (x )
• Therefore, x cannot be a local optimum
• This contradicts our assumption that there exists
  a local maximum that is not a global maximum
• Thus the result is true!!

39
Importance of convex functions in NLP

• Suppose that we have an NLP with the following
  properties
• the feasible region, say S, is convex
• the objective function, say f, is convex
• the objective is to minimize the value of the
  objective function

40
Importance of convex functions in NLP

• Then
• Any local minimum is a global minimum!
• This can be proven mathematically in a similar way