Optimization on Open Sets

  • Till now, we had only talked about optimization on closed sets (remember, Bolzano-Weierstrass assumes the feasible region to be closed and bounded). Now, we address optimization problem on open sets.
  • Find the \(4^{th}\) lecture notes linked: SC 607 Lecture Notes 4
  • Taylor’s Theorem was stated and used to get rate of convergence. As a result, optimal solution of a differentiable real valued function on an open set must have derivative \(=0\).
  • Also use Taylor’s theorem to get quadratic convergence when the function above is twice differentiable. This gives a condition on the hessian to be positive semi-definite at the optimal point. This condition can be strengthened to give a sufficient condition that: if at the optimal solution, the hessian is strictly positive definite, then we have a local minimum.
  • This shows that we can reduce the search for solutions over the entire feasible region to just the set of points where the derivative of \(f\) vanishes.
  • Next time we shall link unconstrained minimization to optimization over open sets.