Optimization

Optimization is the process of finding the input to a function that minimizes or maximizes it.

Objective Function

The quantity being optimized, $f(x)$, is the objective function or merit function. The quantities free to adjust, $x$, are the design variables.

Turning Points

At a turning point, $f'(x) = 0$. The second derivative $f''(x)$ determines the type

• Maximum
  $f''(x) < 0$.
• Minimum
  $f''(x) > 0$.

One-Dimensional Optimization

One-dimensional optimization finds $x$ such that $f'(x) = 0$.

Constrained and Unconstrained Optimization

• Unconstrained optimization
  No restrictions are placed on the design variables. See steepest ascent method.
• Constrained optimization
  Restrictions are placed on the design variables. See constrained optimization.

Local and Global Optima

A function is unimodal iff it has a single optimum. A function with more than 1 local optimum is multimodal.