[Optimal Design] 4. Optimum Design Concepts: Optimality Conditions

On this page

• Introduction to Optimum Design
  • 4. Optimum Design Concepts: Optimality Conditions
    • 4.1 Global and local minima
    • 4.2 Basic calculus concepts

Introduction to Optimum Design

4. Optimum Design Concepts: Optimality Conditions

---

- 이 장의 주요내용:

• 비제약조건 및 제약조건 최적화 문제에서 국소적 및 전역적 최소(최대) 정의

• 비제약조건 최적화 문제에 대한 최적성 조건의 기술

• 제약조건 최적화 문제에 대한 최적성 조건의 기술

• 비제약조건 및 제약조건 문제에서 주어진 점에 대한 최적성 조건의 검토

• 후보 최소점을 위한 1차 최적성 조건의 풀이

• 함수와 설계 최적화 문제의 볼록성 검토

• 제약조건의 변동에 따른 목적함수의 최적값 변화를 연구하기 위한 라그랑지 승수의 활용

---

4.1 Global and local minima

• minimum ← sigular, minima ← plurar

• Global minimum

  $f(x^{\ast })\le f(x)\text{for all}x\text{in the feasible set, }S$

• Local minimum

  $f(x^{\ast })\le f(x)\text{for all}x\text{in a small neighborhood}N\text{of}{x}^{\ast }\text{in the feasible set, }S$

  $N=\{x|x\in S\text{with}||x-x^{\ast }||<\delta \}$ where $\delta$: small real number.

• Existence of a minimum

  • Weierstrass theorem(필요조건, 만족안해도 미니멈 존재할 수 있음)

    $f(x)$ is continuous on a non-empty feasible set S that is closed and bounded, then $f(x)$ has a global minimum is S.

    1. set S is closed $\iff a\le x\le b$

    2. set S is bounded $\iff$ any $x\in S,x^{T}x<c$

       • c는 상수, $x$는 single이면 $x^2<c$

    e.g.,

    [0,1] : closed, bounded.

    [2n, 2n+1], $n\in Z$(정수) : closed, but not bounded ($\because n\to \mathrm{\infty }$)

    (0, 1] : not closed, but bounded

    (2n, 2n+1) : not closed, not bounded

$a\le x\le b$ : closed.

Any $x\in S,x^{T}x<c$ : bounded

→ global minimum exist

4.2 Basic calculus concepts

Vectors

Sets

4.2.1 Gradient vectors (first-order partial derivative of a function)

4.2.2 Hessian matrix (second-order partial derivatives)

• ${\mathrm{\nabla }}^{2}f=\mathbf{H}$

  

  $\mathbf{H}={\mathbf{H}}^T$ (always symmetric)

4.2.3 Taylor’s Expansion

1. single variable $x$

   $$f(x)=f(x^{\ast })+\frac{df(x^{\ast })}{dx}(x-x^{\ast })+\frac{1}{2}\frac{d^{2}f(x^{\ast })}{d{x}^2}(x-x^{\ast }{)}^2+\cdots$$

   Let $x-x^{\ast }=d$

   $$f(x^{\ast }+d)=f(x^{\ast })+\frac{df(x^{\ast })}{dx}d+\frac{1}{2}\frac{d^{2}f(x^{\ast })}{d{x}^2}{d}^2+\cdots$$

2. Two variables, $x_1$ and $x_2$

   $$f(x_1^{\ast }+d_1,x_2^{\ast }+d_2)=f(x_1^{\ast },x_2^{\ast })+\frac{df}{d{x}_1}{d}_1+\frac{df}{d{x}_2}{d}_2+\frac{1}{2}\{\frac{{\partial }^{2}f}{\partial x_1^2}{d}^2+2\frac{{\partial }^{2}f}{\partial x_1\partial x_2}{d}_1{d}_2+\frac{{\partial }^{2}f}{\partial x_2^2}{d}_2^2\}+\cdots$$

   Using summation notations,

   $$=f(x_1^{\ast },x_2^{\ast })+\sum _{i=1}^2\frac{\partial f}{\partial x_i}{d}_i+\frac{1}{2}\sum _{i=1}^2\sum _{i=1}^2\frac{{\partial }^{2}f}{\partial x_i\partial x_j}{d}_i{d}_j+\cdots$$

   Using matrix notation,

   $$f({\overline{x}}^{\ast }+\overline{d})=f({\overline{x}}^{\ast })+(\overline{\mathrm{\nabla }}f{)}^T\overline{d}+\frac{1}{2}{\overline{d}}^T\overline{H}\overline{d}+\cdots$$

   Let $\mathrm{\Delta }f=f(\overline{x^{\ast }}+\overline{d})-f(\overline{x^{\ast }})$

   $$\mathrm{\Delta }f=(\overline{\mathrm{\nabla }}f{)}^T\overline{d}+\frac{1}{2}{\overline{d}}^T\overline{H}\overline{d}+\cdots$$

• $\overline{x},\tilde{x}\to$ 벡터나 매트릭스. multiple variable notation.

4.2.4 Quadratic form (Q-form) and definite matrix

1. Q-form

   $$f=\sum _{i=1}^n\sum _{j=1}^n{p}_{xj}{x}_i{x}_j={\overline{x}}^T\overline{P}\overline{x}\leftarrow \text{(Q-form)}$$

   e.g., $f=x_1^2-6x_1{x}_2+9x_2^2$

   $=(x_1{x}_2)\left[\begin{array}{cc}1& -3\\ -3& 9\end{array}\right]\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)={\overline{x}}^T\overline{P}\overline{x}$

2. Positive definiteness (P-D) of Q-form

   P-D if $f(x)>0$ for all non-zero $\tilde{x}$

   • 아래로 볼록! convex

   P-semi D if $f(\tilde{x})\ge 0$ for all $\tilde{x}$

   e.g., $f=\frac{1}{3}{x}^2$ : P-D

   $f>0$ for $x\ne 0$

   $f=0$ iff $x=0$

   • negative는 위로 볼록, minimum 없음.

• How to check P-D of ${\tilde{x}}^T\tilde{P}\tilde{x}$

  1. check eigenvalues of $\tilde{P}$

  2. sylvester’s test