Вы находитесь на странице: 1из 30

EE3216 – ENGINEERING

OPTIMIZATION
HERMINARTO NUGROHO
TEKNIK ELEKTRO – UNIVERSITAS PERTAMINA
2018-2019
Course Objectives

▷ Critical Thinking and Creative Problem Solving


Students will be able to think critically and creatively to find suitable
optimization method for a certain engineering problem.

▷ Optimization Problem Identification


Students will be able to identify or recognize engineering optimization
problems.

▷ Implement Suitable Method


Students will be able to implement suitable method to solve a certain
engineering problem.

▷ Optimization Impact
Students will be able to explain the impact of using optimization concept.
What will we learn?

WEEK 1 Intro to Engineering Optimization.

WEEK 2 Mathematical Frameworks of Optimization.

WEEK 3 Linear Programming.

WEEK 4 Quadratic Programming.

WEEK 5 Non-Linear Optimization without Constraints.

WEEK 6 Non-Linear Optimization with Constraints.

WEEK 7 Global Optimization Methods.

WEEK 8 Mid-term Exam.


4.
QUADRATIC
PROGRAMMING
Quadratic Programming, Lagrange Condition
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Overview

1. Quadratic Programming in Standard Form

2. Quadratic Programming without Constraint

3. Quadratic Programming with Equality


Constraint

4. Examples

5. Summary
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming:
Introductory Example
Tom wants to build a house on his land. His
land is a triangle with dimension 10m and 20m
perpendicular. He wants to build a house with
two square rooms with maximum area as
10
possible. Can you help him find the dimension
of his house?

max 𝑥 2 + 𝑦 2
𝑥,𝑦
𝑥 s.t the constraints
𝑦

20
𝑥 𝑦
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
𝑥1
Let 𝑥 = 𝑥 . Compute 𝑥 𝑇 𝐴𝑥 for the following matrices:
2

a 4 0 b 3 −2
𝐴= 𝐴=
0 3 −2 7

a 𝑥2 4 0 𝑥1 4𝑥1
𝑇
𝑥 𝐴𝑥 = 𝑥1 𝑥 = 𝑥1 𝑥2 = 4𝑥12 + 3𝑥22
0 3 2 3𝑥2

3 −2 𝑥1 3𝑥1 − 2𝑥2
b 𝑥 𝑇 𝐴𝑥 = 𝑥1 𝑥2 = 𝑥1 𝑥2
−2 7 𝑥2 −2𝑥1 + 7𝑥2
= 𝑥1 3𝑥1 − 2𝑥2 + 𝑥2 −2𝑥1 + 7𝑥2
= 3𝑥12 − 4𝑥1 𝑥2 + 7𝑥22
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
𝑥1 𝑥2 4 0 𝑥1 𝑥1 𝑥2 3 −2 𝑥1
0 3 𝑥2 −2 7 𝑥2

4𝑥12 + 3𝑥22 3𝑥12 − 4𝑥1 𝑥2 + 7𝑥22


1. The value in front of 𝑥12 and 𝑥22 is located in diagonal of the matrix 𝐴 (Based
on the configuration of vector 𝑥).

2. The value in front of 𝑥1 𝑥2 is divided evenly and is placed based on the


configuration of vector 𝑥.
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
Let 𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3 , write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥:

𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3

5 𝑥1
𝑥1 𝑥2 𝑥3 3 𝑥2
𝑥3
2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
Let 𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3 , write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥:

𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3

1
5 −
2 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 𝑥2
2
𝑥3
2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
Let 𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3 , write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥:

𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3

1
5 −
2 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 4 𝑥2
2
𝑥3
4 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
Let 𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3 , write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥:

𝑓 𝑥 = 5𝑥12 + 3𝑥22 + 2𝑥32 − 𝑥1 𝑥2 + 8𝑥2 𝑥3

1
5 −
2
0 𝑥1
1
𝑥1 𝑥2 𝑥3 − 3 4 𝑥2
2
𝑥3
0 4 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Function in Matrix


Form
Let 𝑓 𝑥 = 2𝑥12 + 9𝑥32 − 4𝑥1 𝑥2 + 2𝑥1 𝑥3 + 5𝑥1 − 2𝑥2 ,

1
write 𝑓(𝑥) as 𝑥 𝑇 𝐴𝑥 − 𝑅𝑇 𝑥:
2

𝑓 𝑥 = 2𝑥12 + 9𝑥32 − 4𝑥1 𝑥2 + 2𝑥1 𝑥3 + 5𝑥1 − 2𝑥2

1
1 −1
1 2 𝑥1 𝑥1
𝑥 𝑇 𝐴𝑥 − 𝑅𝑇 𝑥 = 𝑥1 𝑥2 𝑥3 −1 0 0 𝑥2 − −5 2 0 𝑥2
2 1 9 𝑥3 𝑥3
0
2 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming in
Standard Form
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2

Subject to: ℎ 𝑥 =0
𝑔(𝑥) ≤ 0
Where 𝑄 is a symmetric positive definite matrix.
Quadratic Programming : Quadratic Objective Function and
Linear Constraints
Note: All quadratic programming problems should be modified such that it
look like the equation above
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming without


Constraint Problem
Quadratic Programming without Constraint Problem Formulation:

1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Quadratic function
𝑥 2
Where 𝑄 is a symmetric positive definite matrix.

For an unconstrained optimization problem, the condition for optimality is:

Zero Gradient Condition :

∇𝑓 𝑥 = 0

However, how can we find the gradient of a Quadratic Function in Matrix Form?
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Derivative of Quadratic Function


in Matrix Form
Let the quadratic function be represented by

𝑞 𝑥 = 𝑥 𝑇 𝑄𝑥
Then, the gradient of 𝑞(𝑥), or 𝛻𝑞(𝑥) is

∇𝑞 𝑥 = 𝑥 𝑇 𝑄 + 𝑄 𝑇
For a particular case in which 𝑄 is symmetric matrix, then

∇𝑞 𝑥 = 2𝑥 𝑇 𝑄
Because, if 𝑄 is symmetric, then 𝑄𝑇 = 𝑄.
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Derivative of Quadratic Function


in Matrix Form
Let the quadratic function be represented by
1 𝑇
𝑞 𝑥 = 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
2
With 𝑄 is symmetric, then, the gradient of 𝑞(𝑥), or 𝛻𝑞(𝑥) is

∇𝑞 𝑥 = 𝑥 𝑇 𝑄 − 𝑅𝑇
This result can be used to solve Quadratic Programming without constraint.

Note: You can choose which solution you want, as long as the dimension
of the gradient is consistent
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming without


Constraint Problem
Quadratic Programming without Constraint Problem Formulation:

1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Quadratic function
𝑥 2
Where 𝑄 is a symmetric positive definite matrix.

For an unconstrained optimization problem, the condition for optimality is:

Zero Gradient Condition :

∇𝑓 𝑥 ∗ = 𝑥 𝑇 𝑄 − 𝑅𝑇 = 0

𝑥 𝑇 𝑄 = 𝑅𝑇 ⟹ 𝑥 𝑇 = 𝑅𝑇 𝑄 −1 𝑥 = 𝑄−1 𝑅
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming without


Constraint Problem (Example)
Solve the following Quadratic Programming without Constraint Problem

min 𝑥12 + 𝑥22 + 𝑥32 + 4𝑥1 + 6𝑥3 + 20


𝑥

Quadratic Programming in Standard Form:

1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2
Where:

2 0 0
𝑄= 0 2 0 𝑅𝑇 = −4 0 −6 𝑐 = 20
0 0 2
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming without


Constraint Problem (Example)
Quadratic Programming in Standard Form:

1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
Where:
𝑥 2

2 0 0
𝑄= 0 2 0 𝑅𝑇 = −4 0 −6 𝑐 = 20
0 0 2
Then: 𝑥 ∗ = 𝑄−1 𝑅
The minimum value of
1/2 0 0 −4 −2 objective function:
𝑥∗ = 0 1/2 0 0 = 0 𝑞(𝑥 ∗ ) = 7
0 0 1/2 −6 −3
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming with


Equality Constraint
Quadratic Programming with Equality Constraint Problem:
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2
Subject to: ℎ 𝑥 = 𝐴𝑥 − 𝑏 = 0
Where 𝑄 is a symmetric positive definite matrix.

For an equality-constrained optimization problem, the condition for


optimality is:

Lagrange Condition : There exists a vector 𝜆, such that:

∇𝑓 𝑥 − 𝜆𝑇 ∇ℎ 𝑥 = 0
ℎ 𝑥 =0
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming with


Equality Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐
𝑥 2
Subject to: ℎ 𝑥 = 𝐴𝑥 − 𝑏 = 0

Lagrange Condition : There exists a vector 𝜆, such that:

∇𝑓 𝑥 − 𝜆T ∇ℎ 𝑥 = 0
𝑥 𝑇 𝑄 − 𝑅𝑇 − 𝜆𝑇 𝐴 = 0
𝑥 𝑇 𝑄 = 𝑅𝑇 + 𝜆𝑇 𝐴

𝑥 𝑇 = 𝑅𝑇 + 𝜆𝑇 𝐴 𝑄 −1
or equivalently
𝑥 = 𝑄−1 𝑅 + 𝐴𝑇 𝜆
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming with


Equality Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Subject to: ℎ 𝑥 = 𝐴𝑥 − 𝑏 = 0
𝑥 2

Lagrange Condition : There exists a vector 𝜆, such that:

𝑥 = 𝑄 −1 𝑅 + 𝐴𝑇 𝜆

Now, how can we find 𝜆 ?

𝐴𝑥 = 𝐴𝑄−1 𝑅 + 𝐴𝑇 𝜆

𝑏 = 𝐴𝑄 −1 𝑅 + 𝐴𝑇 𝜆 = 𝐴𝑄 −1 𝑅 + 𝐴𝑄 −1 𝐴𝑇 𝜆

𝜆 = 𝐴𝑄−1 𝐴𝑇 −1 (𝑏 − 𝐴𝑄−1 𝑅)
1. QP in Standard 2. QP without 3. QP with Equality
Form Constraint
4. Examples 5. Summary
Constraint

Quadratic Programming with


Equality Constraint
1 𝑇
min 𝑥 𝑄𝑥 − 𝑅𝑇 𝑥 + 𝑐 Subject to: ℎ 𝑥 = 𝐴𝑥 − 𝑏 = 0
𝑥 2

Lagrange Condition : There exists a vector 𝜆, such that:

𝑥 = 𝑄 −1 𝑅 + 𝐴𝑇 𝜆

𝜆 = 𝐴𝑄 −1 𝐴𝑇 −1 (𝑏 − 𝐴𝑄 −1 𝑅)
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary

Examples
1 min 2𝑥 2 + 3𝑦 2 + 20 Answer:
𝑥,𝑦

Subject to: 𝑥 + 𝑦 = 5 𝑥∗ = 3
𝑦∗ = 2
𝑓 𝑥 ∗ , 𝑦 ∗ = 50

2 min 3𝑥 2 + 4𝑦 2 − 705 Answer:


𝑥,𝑦

Subject to: 𝑥 ∗ = 12
𝑥 + 𝑦 − 21 = 0 𝑦∗ = 9
𝑓 𝑥 ∗ , 𝑦 ∗ = 51
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary

Examples
3 min 𝑥 2 + 2𝑦 2 + 2𝑥𝑦 + 3 Answer:
𝑥,𝑦

Subject to: 2𝑥 + 3𝑦 = 5 𝑥∗ = 1
𝑦∗ = 1
𝑓 𝑥∗, 𝑦∗ = 8

4 min 3𝑥 2 + 𝑦 2 + 2𝑥𝑦 Answer:


𝑥,𝑦

Subject to: 𝑥 ∗ = −1
𝑥 + 3𝑦 − 11 = 0 𝑦∗ = 4
𝑓 𝑥 ∗ , 𝑦 ∗ = 11
2. LP in
1. Intro to LP
Standard Form
3. Simplex Method 4. Examples 5. Summary

Summary

▷Quadratic Programming : Quadratic


objective function and Linear constraints.

▷Gradient condition is a good way to solve


Quadratic Programming without
constraint problems

▷Lagrange Condition is a good way to


solve Quadratic Programming with
equality constraint problems.
1. Optimization 2. Optimization 3. Optimization 4. Optimization
Concept History
5. Examples
Objectives Problem

HOMEWORK 4
1 min 𝑥 2 + 𝑦 2 + 𝑧 2 + 4𝑥 + 6𝑧 + 20
𝑥,𝑦

Subject to: 𝑥+𝑦 =0

Hint: 𝑓 𝑥 ∗, 𝑦 ∗, 𝑧 ∗ = 9
References

▷ Edwin K. P. Chong, Stanislaw H. Zak , “An Introduction to


Optimization", Wiley, 2013.
Thanks!
Any questions?

Вам также может понравиться