EMT 212 Computational Engineering

Chapter 1 Optimization Methods

1
 Optimization Methods
1.1 Introduction

Optimization is the act of attaining the best possible

solution under the given constraints. It is the process of
finding the optimal condition that give the maximum or
minimum of a function.

K.H. Yu 2
 Optimization Methods
1.1 Introduction

Example of an optimization problem

You are given 40 m of fencing and you want to enclose

a space for a garden. Find the largest area that can be
enclosed with this much fencing.

K.H. Yu 3
 Optimization Methods
1.1 Introduction

Solution
In mathematical terms,
𝑢 𝑥, 𝑦 = 𝑥𝑦 (1.1)
The constraint is given as
2𝑥 + 2𝑦 = 40 (1.2)
Express (1.2) in terms of 𝑦,
𝑦 = 20 − 𝑥 (1.3)
Substitute (1.3) into (1.1),
𝑢 𝑥 = 𝑥(20 − 𝑥) (1.4)

For optimization problem, the purpose is to find the

optimal solution, i.e., maximum of 𝑢(𝑥).

K.H. Yu 4
 Optimization Methods
1.1 Introduction

𝑢 = 𝑥𝑦

K.H. Yu 5
 Optimization Methods
1.2 1D Unconstrained Optimization
1.2.1 Newton’s Method

To find the root of a function, use the function 𝑢(𝑥) and its
first derivative 𝑢′(𝑥)
𝑢(𝑥𝑖 )
𝑥𝑖+1 = 𝑥𝑖 − (1.5)
𝑢′(𝑥𝑖 )

To find the local maximum or minimum of a function, use

the first 𝑢′(𝑥) and second derivative 𝑢′′(𝑥) of the function

𝑢′(𝑥𝑖 )
𝑥𝑖+1 = 𝑥𝑖 − (1.6)
𝑢′′(𝑥𝑖 )

K.H. Yu 6
 Optimization Methods

Example 1.1

Find the a) local maximum and b) the root of

𝑢 𝑥 = 20𝑥 − 𝑥 2 From (1.4)

using the Newton’s method.

K.H. Yu 7
 Optimization Methods

Example 1.1
Solution
Find the first derivative and the second derivative of the
function
𝑢 𝑥 = 20𝑥 − 𝑥 2 (1.7)
𝑢′ 𝑥 = 20 − 2𝑥 (1.8)
𝑢′′ 𝑥 = −2 (1.9)

K.H. Yu 8
 Optimization Methods

Example 1.1
Solution
a) To find the local maximum point,
First iteration
Assume the starting point at 𝑥0 = 5
𝑢′ 5 = 20 − 2 5 = 10
𝑢′′ 5 = −2
𝑢′(𝑥0 )
Based on (1.6), 𝑥1 = 𝑥0 −
𝑢′′(𝑥0 )
10
𝑥1 = 5 − = 10
−2

K.H. Yu 9
 Optimization Methods

Example 1.1
Solution
Second iteration

At 𝑥1 = 10
𝑢′ 10 = 20 − 2 10 = 0
𝑢′′ 10 = −2
𝑢′(𝑥1 )
Based on (1.6), 𝑥2 = 𝑥1 −
𝑢′′(𝑥1 )
0
𝑥2 = 10 − = 10
−2

K.H. Yu 10
 Optimization Methods

Example 1.1
Solution
b) To find the root,
First iteration
Assume the starting point at 𝑥0 = 5
2
𝑢 5 = 20 5 − 5 = 75
𝑢′ 5 = 20 − 2 5 = 10

Based on (1.5), 𝑢(𝑥0 )

𝑥1 = 𝑥0 −
𝑢′(𝑥0 )
75
𝑥1 = 5 − = −2.5
10

K.H. Yu 11
 Optimization Methods

Example 1.1
Solution
Second iteration

At 𝑥1 = −2.5
𝑢 −2.5 = 20 −2.5 − −2.5 2 = −56.25
𝑢′ 5 = 20 − 2 −2.5 = 25
𝑢(𝑥1 )
Based on (1.5), 𝑥2 = 𝑥1 −
𝑢′(𝑥1 )
−56.25
𝑥2 = −2.5 − = −0.25
25

K.H. Yu 12
 Optimization Methods

Example 1.1
Solution
Third iteration

At 𝑥2 = −0.25
2
𝑢 −0.25 = 20 −0.25 − −0.25 = −5.0625
𝑢′ −0.25 = 20 − 2 −0.25 = 20.5
𝑢(𝑥2 )
Based on (1.5), 𝑥3 = 𝑥2 −
𝑢′(𝑥2 )
−5.0625
𝑥3 = −0.25 − = −0.003
20.5

K.H. Yu 13
 Optimization Methods
1.3 Linear Programming
1.3.1 Overview

• Refers to linear optimization and NOT computer coding.

• Optimization process comprises of
❖A linear objective function
❖A set of (linear) constraints
❖Constraints of individuals parameters

Purpose

A method to attain the best outcome in a mathematical

model with given constraints.

K.H. Yu 14
 Optimization Methods
1.3 Linear Programming
1.3.2 General Mathematical Form

Objective
𝑧 = 𝑎1 𝑥1 + 𝑎2 𝑥2 + ⋯ + 𝑎𝑛 𝑥𝑛 = 𝐚 ∙ 𝐱
function
𝑎𝑖1 𝑥1 + 𝑎𝑖2 𝑥2 + ⋯ + 𝑎𝑖𝑛 𝑥𝑛 ≤ 𝑏𝑖
Set of 𝑎𝑗1 𝑥1 + 𝑎𝑗2 𝑥2 + ⋯ + 𝑎𝑗𝑛 𝑥𝑛 ≥ 𝑏𝑗
constraints
𝑎𝑘1 𝑥1 + 𝑎𝑘2 𝑥2 + ⋯ + 𝑎𝑘𝑛 𝑥𝑛 = 𝑏𝑘

Constraints of
individual 𝑏𝑖 , 𝑏𝑗 , 𝑏𝑘 ≥ 0
parameters

K.H. Yu 15
 Optimization Methods
1.3 Linear Programming
1.3.3 Graphical Method

• Graphical method is a simple method in linear

programming for solving optimization problems
involving one or two variables.
• For two optimization variables, best possible
solutions can be attained by drawing the contours
of objective function and constraint functions.
• The maximum (or minimum) can be simply read
from the graph.

K.H. Yu 16
 Optimization Methods

Example 1.2

Suppose the objective function 𝑧 is given as

𝑧 𝑥1 , 𝑥2 = 95𝑥1 + 150𝑥2 Objective function

and the constraints are specified as

5𝑥1 + 10𝑥2 ≤ 30
20𝑥1 + 12𝑥2 ≤ 72 Set of constraints

𝑥1 ≤ 4
𝑥2 ≤ 3
Constraints of
𝑥1 , 𝑥2 ≥ 0 individual
parameters

K.H. Yu 17
 Optimization Methods

Example 1.2
MATLAB command can be used to generate the contour
clear all
close all

x1=[0:0.1:6];
x2=[0:0.1:6];

for i=1:length(x1)
for j=1:length(x2)
z(i,j)=95*x1(i)+150*x2(j);
end
end

figure(1)
contour(z,[150 300 450 600 750 900 1050 1200
1350],'ShowText','on','LineColor','b')
xlabel('$x_{1}$', 'Interpreter', 'Latex','FontSize', 16)
ylabel('$x_{2}$', 'Interpreter', 'Latex','FontSize', 16)

K.H. Yu 18
 Optimization Methods

Example 1.2

K.H. Yu 19
 Optimization Methods

Example 1.2

Continue execute the subsequent MATLAB command

%Constraint #1
x2_c1=(30-5*x1)/10;
%Constraint #2
x2_c2=(72-20*x1)/12;

hold on
plot(x1,x2_c1,'r')
hold on
plot(x1,x2_c2,'r')
hold on
plot(4*ones(10,1),linspace(0,6,10),'r')
hold on
plot(linspace(0,6,10),3*ones(10,1),'r')
xlim([0,6])
ylim([0,6])

K.H. Yu 20
 Optimization Methods

Example 1.2

K.H. Yu 21
 Optimization Methods

Example 1.2

Based on the
graph, the
Maximum point maximum
𝑥2,𝑚𝑎𝑥 value of z is
approximately
500.

𝑥1,𝑚𝑎𝑥

K.H. Yu 22
 Optimization Methods

Example 1.2

Alternatively, a shorter MATLAB command can be used.

clear all
close all

x1 = 0:6; % range for graph

x2a = max(3 - 0.5*x1,0);
x2b = max((72-20*x1)/12,0);
x2c = max(3*ones(1,length(x1)),0);
ytop = min([x2a; x2b; x2c]);
area(x1,ytop);
xlabel('$x_{1}$', 'Interpreter', 'Latex','FontSize', 16)
ylabel('$x_{2}$', 'Interpreter', 'Latex','FontSize', 16)

K.H. Yu 23
 Optimization Methods

Example 1.2

Maximum point

K.H. Yu 24
 Optimization Methods
1.3 Linear Programming
1.3.4 Simplex Method

• Another optimization method that is frequently

used is the simplex method.

• Rewrite the mathematical statement with

inequalities with the corresponding statement with
equalities.

i) For set of constraints involving ≤, slack variable

is added.
ii) For set of constraints involving ≥, surplus variable
(subtracted from the inequalities) is introduced.

K.H. Yu 25
 Optimization Methods

Example 1.3

Suppose the objective function 𝑧 is given as

𝑧 𝑥1 , 𝑥2 = 95𝑥1 + 150𝑥2
and the constraints are specified as
5𝑥1 + 10𝑥2 ≤ 30
20𝑥1 + 12𝑥2 ≤ 72 Set of constraints
𝑥1 ≤ 4 with inequalities
𝑥2 ≤ 3

𝑥1 , 𝑥2 ≥ 0

K.H. Yu 26
 Optimization Methods

Example 1.3
Solution
With four constraint equations involving ≤, 4 slack variables
(i.e., 𝑆1 , 𝑆2 , 𝑆3 and 𝑆4 ). Now we have 6 unknowns.

5𝑥1 + 10𝑥2 + 𝑆1 = 30 (1.10)

20𝑥1 + 12𝑥2 + 𝑆2 = 72 (1.11)
𝑥1 + 𝑆3 = 4 (1.12)
𝑥2 + 𝑆4 = 3 (1.13)
Thus, the objective function then become
𝑧 − 95𝑥1 − 150𝑥2 + 0𝑆1 + 0𝑆2 + 0𝑆3 + 0𝑆4 = 0 (1.14)

K.H. Yu 27
 Optimization Methods

Example 1.3
Solution
STEP ONE: Create a tableau

K.H. Yu 28
 Optimization Methods

Example 1.3
Solution
STEP TWO: Make the column with the largest negative coefficient in
the z row as the entering variable.

Pivot Entering variable

column

K.H. Yu 29
 Optimization Methods

Example 1.3
Solution
STEP THREE: Make the smallest positive intercept as the leaving
variable.
Leaving variable
Intercept
Pivot 30/10
row 72/12

3/1

Pivot
column Remark: Intercept is the ratio of the
RHS over the entering variable column

K.H. Yu 30
 Optimization Methods

Example 1.3
Solution
STEP FOUR: Using the pivot, perform Gaussian
elimination on the other rows.
From (1.10) 5𝑥1 + 10𝑥2 + 𝑆1 = 30
Rearranging,
1 1
𝑥2 = 3 − 𝑆1 − 𝑥1 (1.15)
10 2
Substituting (1.15) into (1.11),
1 1
20𝑥1 + 12 3 − 𝑆1 − 𝑥1 + 𝑆2 = 72
10 2
12
14𝑥1 − 𝑆1 + 𝑆2 = 36 (1.16)
10
K.H. Yu 31
 Optimization Methods

Example 1.3
Solution
Substituting (1.15) into (1.13),
1 1
3 − 𝑆1 − 𝑥1 + 𝑆4 = 3
10 2
1 1
− 𝑆1 − 𝑥1 + 𝑆4 = 0 (1.17)
10 2
Substituting (1.15) into (1.14),
1 1
𝑧 − 95𝑥1 − 150 3 − 𝑆1 − 𝑥1 + 0𝑆1 + 0𝑆2 + 0𝑆3 + 0𝑆4 = 0
10 2

𝑧 − 20𝑥1 + 15𝑆1 = 450 (1.18)

K.H. Yu 32
 Optimization Methods

Example 1.3
Solution
Using (1.16) – (1.18), the tableau becomes
Leaving variable

K.H. Yu 33
 Optimization Methods

Example 1.3
Solution
Repeat the Gaussian elimination
12
From (1.16) 14𝑥1 − 𝑆1 + 𝑆2 = 36
10
Rearranging,
36 12 1
𝑥1 = + 𝑆1 − 𝑆2 (1.19)
14 140 14
Substituting (1.19) into (1.15),
1 1 36 12 1
𝑥2 = 3 − 𝑆1 − + 𝑆1 − 𝑆2
10 2 14 140 14
20 1 24
𝑥2 + 𝑆1 − 𝑆2 = (1.20)
140 28 14
K.H. Yu 34
 Optimization Methods

Example 1.3
Solution
Substituting (1.19) into (1.12),
36 12 1
+ 𝑆1 − 𝑆2 + 𝑆3 = 4
14 140 14
12 1 20
𝑆1 − 𝑆2 + 𝑆3 = (1.21)
140 14 14

Substituting (1.19) into (1.17),

1 1 36 12 1
− 𝑆1 − + 𝑆1 − 𝑆2 + 𝑆4 = 0
10 2 14 140 14
20 1 18
− 𝑆1 + 𝑆2 + 𝑆4 = (1.22)
140 28 14

K.H. Yu 35
 Optimization Methods

Example 1.3
Solution
Substituting (1.19) into (1.18),
36 12 1
𝑧 − 20 + 𝑆1 − 𝑆2 + 15𝑆1 = 450
14 140 14
93 20 7020
𝑧 + 𝑆1 + 𝑆2 = (1.23)
7 14 14

K.H. Yu 36
 Optimization Methods

Example 1.3

Using (1.19) – (1.23), the tableau becomes

Maximum value of z

K.H. Yu 37
 Optimization Methods

Example 1.3
Alternatively, MATLAB command as stated
below can be executed to yield similar result.
clear all
close all

z=[-95 -150];
A=[5 10; 20 12;1 0;0 1;-1 0;0 -1];
b=[30; 72; 4; 3; 0; 0];

var=linprog(z, A, b);
profit=-z*var

ANSWER: 501.42 Remark: linprog function

can be used to execute
simplex method.

K.H. Yu 38
 Optimization Methods

Example 1.4

A company owner has RM100,000 to spend in a month to

produce two products, i.e., Product A and Product B. The
cost to produce Product A is RM50 per piece and for
Product B is RM80 per piece. The profit for each Product
A is RM10 and for Product B is RM20. The average
production time for a single Product A and Product B are
10 minutes and 30 minutes, respectively. Assume that
there are 2 production lines running everyday
simultaneously in one 8-hour day, calculate the maximum
profit in 30 days.

K.H. Yu 39
 Optimization Methods

Example 1.4
Solution
Objective function 𝑧 𝑎, 𝑏 = 10𝑎 + 20𝑏

and the constraints are specified as

50𝑎 + 80𝑏 ≤ 100,000

10𝑎 + 30𝑏 ≤ 28,800
𝑎, 𝑏 ≥ 0

K.H. Yu 40
 Optimization Methods

Example 1.4
Solution

Rearranging,
𝑧 − 10𝑎 − 20𝑏 = 0
50𝑎 + 80𝑏 ≤ 100,000
10𝑎 + 30𝑏 ≤ 28,800
−𝑎 ≤ 0
−𝑏 ≤ 0

K.H. Yu 41
 Optimization Methods

Exercise 1
MATLAB command as stated below can be
used to yield the result.
clear all
close all

z=[-10 -20];
A=[50 80; 10 30;-1 0;0 -1];
b=[100000; 28800; 0; 0];

var=linprog(z, A, b);
profit=-z*var

ANSWER: RM22,514

K.H. Yu 42
 Optimization Methods

Exercise 1
A local car manufacturer company produces three models
of cars (Model A, Model B and Model C). The net profit for
Model A is RM8,000, Model B is RM5,000 and Model C is
RM4,000. The fuel consumptions are 16 km per liter, 19
km per liter and 20 km per liter for Model A, B and C,
respectively. Government is imposing that energy
efficiency vehicles concept that requires the average car
produced to be at least having 18 km per liter. In a day,
only 5 cars of Model A can be manufactured. Determine
the optimal solution to maximize the profit in a day.
Solve the following problem with graphical method,
simplex method and compare with the MATLAB solution.
ANSWER: RM90,000

K.H. Yu 43