# 15'053 Thursday, April 25 - PowerPoint PPT Presentation

PPT – 15'053 Thursday, April 25 PowerPoint presentation | free to download - id: 155f22-MjFiZ

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## 15'053 Thursday, April 25

Description:

### 1. 15.053 Thursday, April 25. Nonlinear Programming Theory. Separable programming ... Convexity and Extreme Points. We say that a set S is convex, if for every ... – PowerPoint PPT presentation

Number of Views:43
Avg rating:3.0/5.0
Slides: 48
Provided by: ann2
Category:
Tags:
Transcript and Presenter's Notes

Title: 15'053 Thursday, April 25

1
15.053 Thursday, April 25
• Nonlinear Programming Theory
• Separable programming

Handouts Lecture Notes
2
Difficulties of NLP Models
Linear Program
Nonlinear Programs
3
Graphical Analysis of Non-linear programs in two
dimensions An example
• Minimize
• subject to

(x - 8)2 (y - 9)2 49 x 2
x 13 x y 24
4
Where is the optimal solution?
Note the optimal solution is not at a corner
point. It is where the isocontour first hits the
feasible region.
5
Another example
Minimize (x-8)2 (y-8)2
Then the global unconstrained minimum is
also feasible.
The optimal solution is not on the boundary
of the feasible region.
6
Local vs. Global Optima
Defn Let x be a feasible solution, then
x is a global max if f(x) f(y) for every
feasible y.
x is a local max if f(x) f(y) for every
feasible y sufficiently close to x (i.e., xj-e
yj xj e for all j and some small e).
There may be several locally optimal solutions.
7
When is a locally optimal solution also globally
optimal?
• We are minimizing. The objective
• function is convex. The feasible
• region is convex.

8
Convexity and Extreme Points
We say that a set S is convex, if for every two
points x and y in S, and for every real number ?
in 0,1, ?x (1-?)y e S.
The feasible region of a linear program is
convex.
We say that an element w e S is an extreme point
(vertex, corner point), if w is not the midpoint
of any line segment contained in S.
9
• Recognizing convex feasible regions
• If all constraints are linear, then the
• feasible region is convex
• The intersection of convex regions is
• convex
• If for all feasible x and y, the
• midpoint of x and y is feasible, then
• the region is convex (except in
• totally non-realistic examples. )

10
Which are convex?
11
Convex Functions Convex Functions f(l y (1-
l)z) l f(y) (1- l)f(z) for every y and z and
for 0 l 1. e.g., f((yz)/2) f(y)/2
f(z)/2 We say strict convexity if sign is lt
for 0lt ? lt1.
Line joining any points is above the curve
12
Convex Functions Convex Functions f(l y (1-
l)z) l f(y) (1- l)f(z) for every y and z and
for 0 l 1. e.g., f((yz)/2) f(y)/2
f(z)/2 We say strict convexity if sign is lt
for 0lt ? lt1.
Line joining any points is above the curve
13
Classify as convex or concave or both or neither.
14
What functions are convex?
• f(x) 4x 7 all linear functions
• f(x) 4x2 13 some quadratic
functions
• f(x) ex
• f(x) 1/x for x gt 0
• f(x) x
• f(x) - ln(x) for x gt 0
• Sufficient condition f(x) gt 0 for all x.

15
What functions are convex?
• If f(x) is convex, and g(x) is convex.
• Then so is
• h(x) a f(x) b g(x) for agt0, bgt0.
• If y f(x) is convex, then
• (x,y) f(x) y is a convex set

16
Local Maximum (Minimum) Property
• A local max of a concave function on a convex
• feasible region is also a global max.
• A local min of a convex function on a convex
feasible
• region is also a global min.
• Strict convexity or concavity implies that the
global
• optimum is unique.
• Given this, we can exactly solve
• Maximization Problems with a concave
objective
• function and linear constraints
• Minimization Problems with a convex
objective
• function and linear constraints

17
Which are convex feasible regions?
(x, y) y x2 2 (x, y) y x2 2 (x,y) y
x2 2
18
More on local optimality
• The techniques for non-linear optimization
• minimization usually find local optima.
• This is useful when a locally optimal
• solution is a globally optimal solution
• It is not so useful in many situations.
• Conclusion if you solve an NLP, try to
• find out how good the local optimal
• solutions are.

19
Solving NLPs by Excel Solver
20
Finding a local optimal for a single variable NLP
Solving NLP's with One Variable
max f(?) s.t. a ?
b Optimal solution is either
a boundary point or satisfies
f'(?) 0 and f ?(?) lt 0.
21
Solving Single Variable NLP (contd.)
If f(?) is concave (or simply unimodal) and
differentiable
max f(?) s.t. a ? b
Bisection (or Bolzano) Search
• Step 1. Begin with the region of uncertainty
for ? as a,
• b. Evaluate f ' (?) at the midpoint ?? (ab)/2.
• Step 2. If f ' (??) gt 0, then eliminate the
interval up to ??.
• If f'(??) lt 0, then eliminate the interval beyond
??.
• Step 3. Evaluate f'(?) at the midpoint of the
new
uncertainty
• is sufficiently small.

22
• Unimodal Functions
• A single variable function f is
• unimodal if there is at most one local
• maximum (or at most one local

minimum) .
23
Other Search Techniques
• Instead of taking derivatives (which may be
• computationally intensive), use two function
• evaluations to determine updated interval.
• Fibonacci Search
• Step 1. Begin with the region of uncertainty
for ? as a,
• b. Evaluate f (?1) and f (?2) for 2
symmetric points
• ?1lt?2.
• Step 2. If f (?1) f (?2), then eliminate the
interval up t?1.
• If f (?1) gt f (?2), then eliminate the
interval beyond ?2.
• Step 3. Select a second point symmetric to the
point
• already in the new interval, rename these
points ?1 and
• ?2 such that ?1lt?2 and evaluate f (?1) and f
(?2). Return
• to Step 2 until the interval is sufficiently
small.

24
On Fibonacci search
1, 1, 2, 3, 5, 8, 13, 21, 34 At iteration
1, the length of the search interval is the
kth fibonacci number for some k At iteration
j, the length of the search interval is the
k-j1 fibonacci number. The technique converges
to the optimal when the function is unimodal.
25
Finding a local maximum using Fibonacci Search.
Length of search
interval
Where the maximum may be
26
The search finds a local maximum, but not
necessarily a global maximum.
27
The search finds a local maximum, but not
necessarily a global maximum.
28
Number of function evaluations in Fibonacci Search
• As new point is chosen symmetrically, the
length lk of
• successive search intervals is given by lk
lk1 lk2 .
• Solving for these lengths given a final
interval length of
• 1, ln 1, gives the Fibonacci numbers 1, 2,
3, 5, 8, 13, 21,
• 34,
• Thus, if the initial interval has length 34, it
takes 8
• function calculations to reduce the interval
length to 1.
• Remark if the function is convex or unimodal,
then
• fibonacci search converges to the global
maximum

29
Separable Programming
• Separable programs have the form

Max
st
Each variable xj appears separately, one in each
function gij and one in each function fj in the
objective.
Each non-linear function is of a single variable.
30
Examples of Separable Programs
f(x1,x2) x1(30-x1)x2(35-x2)-2x12-3x22
3
f(x1,x2) x15 -18e-x2-4x2
x1
f(x1,x2,x2) lnx15-sinx2-x3e-x3)7x1-4
31
Approximating a non-linear function with a
piecewise linear function.
• Aspect 1. Choosing the approximation.
• Aspect 2. When is the piecewise linear
• approximation a linear program in
• disguise?

32
Approximating a non-linear function of 1 variable
33
Approximating a non-linear function of 1
variable the ? method
Choose different values of x to approximate the
x-axis
Approximate using piecewise linear segments
34
More on the ? method
a1 -3, f(a1) -20 a2 -1 f(a2) -7 1/3
Suppose that for 3 x -1, we represent x
has?1 (-3) ?2 (-1) where?1 ?2 1 and?1,?2 0
Then we approximate f(x) as ?1 (-20) ?2 (-7 1/3)
35
More on the ? method
Suppose that for 1 x 1, we represent x
has?2 (-3) ?3 (-1) where?2 ?3 1 and?2,?3 0
a2 -1, f(a2) -7 1 /3 a3 1 f(a3) -2 2/3
How do we approximate f( ) in this interval?
What if 3 x 1?
36
Almost the ? method
Original problem min x3/3 2x 5 more
terms s.t. -3 x 3 many more constraints
a1 -3 a2 - 1 a3
1 a4 3 f(a1) -20 f(a2)
-7 1/3 f(a3) -2 2/3 f(a4) 4
Approximated problem min?1f(a1) ?2f(a2)
?3f(a3) ?4f(a4) more
linear terms s.t. ?1 ?2 ?3 ?4 1 ? 0
many more constraints
37
Why the approximation is incorrect
Approximated problem min?1f(a1) ?2f(a2)
?3f(a3) ?4f(a4) more
linear terms s.t. ?1?2 ?3?4 1 ? 0
Consider?1 1/2 ?20 ?31/2 ?4 0
The method gives the correct approximation
if only two consecutive?s are positive.
38
Adjacency Condition 1. At most two weights (?s)
are positive 2. If exactly two weights (?s) are
positive, then they are ?j and ?j1 for some
j 3. The same condition applies to every
approximated function.
39
• Approximating a non-linear objective
• function for a minimization NLP.
• original problem minimize f(y) y ? P
• Suppose that y Sj?jaj ,
• where Sj?j 1
and ? gt 0 .
• Approximate f(y).
• minimize Sj?jf(aj) Sj?jaj? P
• Note when given a choice of representing y in
• alternative ways, the LP will choose one that
• leads to the least objective value for the
• approximation.

40
For minimizing a convex function, the ?-method
property.
min Z ?1f(a1) ?2f(a2) ?3f(a3) ?4f(a4)?5f(a5)
s.t. ?1 ?2 ?3
?4 ?5 1 ? 0
other constraints
41
Feasible approximated objective functions without
min Z ?1f(a1) ?2f(a2) ?3f(a3) ?4f(a4)?5f(a5)
s.t. ?1 ?2 ?3
?4 ?5 1 ? 0
other constraints
42
But a minimum in this case always occurs on the
piecewise linear curve.
min Z ?1f(a1) ?2f(a2) ?3f(a3) ?4f(a4)?5f(a5)
s.t. ?1 ?2 ?3
?4 ?5 1 ? 0
other constraints
43
Separable Programming (in the case of linear
constraints)
Max f ( )x s.t. Dx d x ³ 0
• Begin with an NLP
• Transform to Separable
• Approximate using the ? Method

n
Max ? fj (xj) s.t. Dx d x ³ 0
j1
44
• Approximation
• Re-express in terms of ? variables

If the original problem is concave, then you can
drop the adjacency conditions (they are
automatically satisfied.)
45
How can one construct separable functions?
Term Substitution Constraints
Restriction
None
None
None
46
Transformation Examples
Ex (x1 x2 x3)6
Substitute y6 and let y x1 x2 x3
x1x22
let y1
Ex
and let y2
1 x3
47
NLP Summary
• Convex and Concave functions as well as convex
sets
• are important properties
• Bolzano and Fibonacci search techniques
• used to solve single variable unimodal
functions
• Separable Programming
• nonlinear objective and nonlinear
constraints that
• are separable
• General approximation technique