Till now, we had only talked about optimization on closed sets (remember, Bolzano-Weierstrass assumes the feasible region to be closed and bounded). Now, we address optimization problem on open sets.
Taylor’s Theorem was stated and used to get rate of convergence. As a result, optimal solution of a differentiable real valued function on an open set must have derivative \(=0\).
Also use Taylor’s theorem to get quadratic convergence when the function above is twice differentiable. This gives a condition on the hessian to be positive semi-definite at the optimal point. This condition can be strengthened to give a sufficient condition that: if at the optimal solution, the hessian is strictly positive definite, then we have a local minimum.
This shows that we can reduce the search for solutions over the entire feasible region to just the set of points where the derivative of \(f\) vanishes.
Next time we shall link unconstrained minimization to optimization over open sets.