Optimization with Constraints

• Find the \(3^{3d}\) lecture notes linked: SC 607 IITB Lecture Notes 3 (Google Drive link), Lecture Notes 3 (Github link).
• Find the entire lecture notes linked: SC 607 IITB Lecture Notes 1-3 (Google Drive link), Lecture Notes 1-3 (Github link)
• Interior: \(C\) is any set then, \(\dot{C} = \textrm{interior of C} = \underset{S \textrm{ is open} \, \& S \subseteq C}{\bigcup} S\)
• Closure: Let \(C \subseteq \mathbb{R}^n\) be any set. \(\overline{C} = \textrm{closure of C} = \underset{S \textrm{ is closed} \, \& S \supseteq C}{\bigcap} S\)
• Boundary \(\partial C = \overline{C}\setminus \dot{C}\).
• Interior of a set is an open set since it is an arbitrary union of open sets. In fact it is the largest open set contained in \(C\). Closure of a set is a closed set since it is an arbitrary intersection of closed sets. In fact it is the smallest closed set containing \(C\). It follows that if \(C\) is open, \(\dot{C}=C\) and if \(C\) is closed, then \(\overline{C}=C\). This means that if \(C\) is open, \(\partial C\) contains no point of \(C\) and if \(C\) is closed, all points of \(\partial C\) are points of \(C\).
• Local minimum is exactly what the English word says: a point in the domain is a local minimum of the function if there is a neighbourhood in which the function takes values greater than or equal to the value at such a point. That is \(x^* \in S\) is said to be a \(\textbf{local minimum}\) if \(\exists \, r >0\) s.t. \(f(x^*) \le f(x) \quad \forall x \in B(x^*, r) \cap S\)
• Unconstrained minimum and global minimum are the corresponding terms given for the above inequality holding over all of \(\mathbb{R}^n\) or over all of the feasible region respectively. What about constraints while trying to optimize?
• \(\textbf{Goal}: \min f(x)\) s.t. \(g_i(x) \le 0 \quad \forall i= 1, \ldots , m; \\ h_j(x)= 0 \quad \forall j= 1, \ldots , p\) are satisfied.
• Weierstrass Theorem gives existence of a global minimum.
• Isolated local minimum \(\implies\) strict local minimum.
• Geometry of inequality constraints is very different from that of equality constraints. Interior vs Boundary of a set.