Optimization Multi-Dimensional Unconstrained Optimization (Gradient Methods) Examples and Exercises

1
OptimizationMulti-Dimensional Unconstrained
Optimization (Gradient Methods)Examples and
Exercises
2
Example 1

• Determine whether the stationary point of the
  following quadratic functions is a local maxima,
  local minima or saddle point?

• A point x is a stationary point iff
• f '(x) 0 (if f is a function of one
  variable)
• ?f (x) 0 (if f is a function of gt1 variables)

3
Example 1 Solution
We still have to test if the point is a local
maxima, minima or saddle point (continue next
page )
4
Example 1 Solution (Continue)
(ii) ( continue)
5
Example 1 Solution (Continue)
6
Example 1 Solution (Continue)
(continue next page )
7
Example 1 Solution (Continue)
(iv) ( continue from previous slide)
We can verify if a matrix is positive definite by
checking if the determinants of all its upper
left corner sub-matrices are positive.
Since H is neither positive definite nor negative
definite (i.e., indefinite), the stationary point
is a saddle point.
8
Exercise

• For each of the following points, determine
  whether it is a local maxima, local minima,
  saddle point, or not a stationary point of

1. (0, 0)
2. (1, 0)
3. (-1, -1)
4. (1, 1)

9
Solution