Академический Документы
Профессиональный Документы
Культура Документы
OPTIMIZATION
HERMINARTO NUGROHO
TEKNIK ELEKTRO – UNIVERSITAS PERTAMINA
2018-2019
Course Objectives
▷ Optimization Impact
Students will be able to explain the impact of using optimization concept.
What will we learn?
Overview
4. Examples
5. Summary
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
Quadratic Programming:
Introductory Example
Tom wants to build a house on his land. His
land is a triangle with dimension 10m and 20m
perpendicular. He wants to build a house with
two square rooms with maximum area as
10
possible. Can you help him find the dimension
of his house?
max 𝑥 2 + 𝑦 2
𝑥,𝑦
𝑥 s.t the constraints
𝑦
20
𝑥 𝑦
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
a 4 0 b 3 −2
𝐴= 𝐴=
0 3 −2 7
a 𝑥2 4 0 𝑥1 4𝑥1
𝑇
𝑥 𝐴𝑥 = 𝑥1 𝑥 = 𝑥1 𝑥2 = 4𝑥12 + 3𝑥22
0 3 2 3𝑥2
3 −2 𝑥1 3𝑥1 − 2𝑥2
b 𝑥 𝑇 𝐴𝑥 = 𝑥1 𝑥2 = 𝑥1 𝑥2
−2 7 𝑥2 −2𝑥1 + 7𝑥2
= 𝑥1 3𝑥1 − 2𝑥2 + 𝑥2 −2𝑥1 + 7𝑥2
= 3𝑥12 − 4𝑥1 𝑥2 + 7𝑥22
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
5 𝑥1
𝑥1 𝑥2 𝑥3 3 𝑥2
𝑥3
2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1
5 −
2 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 𝑥2
2
𝑥3
2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1
5 −
2 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 4 𝑥2
2
𝑥3
4 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1
5 −
2
0 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 4 𝑥2
2
𝑥3
0 4 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1
write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥 − 𝑅𝑇 𝑥:
2
1
1 −1
1 2 𝑥1 𝑥1
𝑥 𝑇 𝐴𝑥 − 𝑅𝑇 𝑥 = 𝑥1 𝑥2 𝑥3 −1 0 0 𝑥2 − −5 2 0 𝑥2
2 1 9 𝑥3 𝑥3
0
2 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
Quadratic Programming in
Standard Form
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2
Subject to: ℎ 𝑥 =0
𝑔(𝑥) ≤ 0
Where 𝑄 is a symmetric positive definite matrix.
Quadratic Programming : Quadratic Objective Function and
Linear Constraints
Note: All quadratic programming problems should be modified such that it
look like the equation above
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Quadratic function
𝑥 2
Where 𝑄 is a symmetric positive definite matrix.
∇𝑓 𝑥 = 0
However, how can we find the gradient of a Quadratic Function in Matrix Form?
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
𝑞 𝑥 = 𝑥 𝑇 𝑄𝑥
Then, the gradient of 𝑞(𝑥), or 𝛻𝑞(𝑥) is
∇𝑞 𝑥 = 𝑥 𝑇 𝑄 + 𝑄 𝑇
For a particular case in which 𝑄 is symmetric matrix, then
∇𝑞 𝑥 = 2𝑥 𝑇 𝑄
Because, if 𝑄 is symmetric, then 𝑄𝑇 = 𝑄.
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
∇𝑞 𝑥 = 𝑥 𝑇 𝑄 − 𝑅𝑇
This result can be used to solve Quadratic Programming without constraint.
Note: You can choose which solution you want, as long as the dimension
of the gradient is consistent
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Quadratic function
𝑥 2
Where 𝑄 is a symmetric positive definite matrix.
∇𝑓 𝑥 ∗ = 𝑥 𝑇 𝑄 − 𝑅𝑇 = 0
𝑥 𝑇 𝑄 = 𝑅𝑇 ⟹ 𝑥 𝑇 = 𝑅𝑇 𝑄 −1 𝑥 = 𝑄−1 𝑅
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2
Where:
2 0 0
𝑄= 0 2 0 𝑅𝑇 = −4 0 −6 𝑐 = 20
0 0 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
Where:
𝑥 2
2 0 0
𝑄= 0 2 0 𝑅𝑇 = −4 0 −6 𝑐 = 20
0 0 2
Then: 𝑥 ∗ = 𝑄−1 𝑅
The minimum value of
1/2 0 0 −4 −2 objective function:
𝑥∗ = 0 1/2 0 0 = 0 𝑞(𝑥 ∗ ) = 7
0 0 1/2 −6 −3
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
∇𝑓 𝑥 − 𝜆𝑇 ∇ℎ 𝑥 = 0
ℎ 𝑥 =0
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
∇𝑓 𝑥 − 𝜆T ∇ℎ 𝑥 = 0
𝑥 𝑇 𝑄 − 𝑅𝑇 − 𝜆𝑇 𝐴 = 0
𝑥 𝑇 𝑄 = 𝑅𝑇 + 𝜆𝑇 𝐴
𝑥 𝑇 = 𝑅𝑇 + 𝜆𝑇 𝐴 𝑄 −1
or equivalently
𝑥 = 𝑄−1 𝑅 + 𝐴𝑇 𝜆
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
𝑥 = 𝑄 −1 𝑅 + 𝐴𝑇 𝜆
𝐴𝑥 = 𝐴𝑄−1 𝑅 + 𝐴𝑇 𝜆
𝑏 = 𝐴𝑄 −1 𝑅 + 𝐴𝑇 𝜆 = 𝐴𝑄 −1 𝑅 + 𝐴𝑄 −1 𝐴𝑇 𝜆
𝜆 = 𝐴𝑄−1 𝐴𝑇 −1 (𝑏 − 𝐴𝑄−1 𝑅)
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint
𝑥 = 𝑄 −1 𝑅 + 𝐴𝑇 𝜆
𝜆 = 𝐴𝑄 −1 𝐴𝑇 −1 (𝑏 − 𝐴𝑄 −1 𝑅)
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary
Examples
1 min 2𝑥 2 + 3𝑦 2 + 20 Answer:
𝑥,𝑦
Subject to: 𝑥 + 𝑦 = 5 𝑥∗ = 3
𝑦∗ = 2
𝑓 𝑥 ∗ , 𝑦 ∗ = 50
Subject to: 𝑥 ∗ = 12
𝑥 + 𝑦 − 21 = 0 𝑦∗ = 9
𝑓 𝑥 ∗ , 𝑦 ∗ = 51
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary
Examples
3 min 𝑥 2 + 2𝑦 2 + 2𝑥𝑦 + 3 Answer:
𝑥,𝑦
Subject to: 2𝑥 + 3𝑦 = 5 𝑥∗ = 1
𝑦∗ = 1
𝑓 𝑥∗, 𝑦∗ = 8
Subject to: 𝑥 ∗ = −1
𝑥 + 3𝑦 − 11 = 0 𝑦∗ = 4
𝑓 𝑥 ∗ , 𝑦 ∗ = 11
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary
Summary
HOMEWORK 4
1 min 𝑥 2 + 𝑦 2 + 𝑧 2 + 4𝑥 + 6𝑧 + 20
𝑥,𝑦
Hint: 𝑓 𝑥 ∗, 𝑦 ∗, 𝑧 ∗ = 9
References