6BV04

DOE: Optimization
Response Surface Methods

department of mathematics and computer science

Contents

Optimisation steps
Box method
Steepest ascent method
Practical example
Response surface designs
Multiple responses
EVOP
Software
Literature

department of mathematics and computer science

Optimisation steps

Optimisation is achieved by going through the

following phases:
screening (determine which factors really influence
the outcome; tool: screening designs like fractional
factorial)
improvement (approach optimum by repeated
change of factor settings; tools: Box/simplex or
steepest ascent approach)
determination of optimum (find optimal settings of
factor settings; tool: response surface designs like
CCD or Box-Behnken + analysis of response surface
using eigenvalues)

department of mathematics and computer science


optimum

improvement

current settings

department of mathematics and computer science


Regression models used in optimisation
Statistical techniques for optimisation assume the following (often
reasonably satisfied in practice):
Far away from the optimum a first order model often suffices.
for example:
Y = 0 + 1x1 + 2x2 +
Near the optimum often a quadratic (second order) model
suffices. For example:
Y = 0 + 1x1 + 2x2 + 12x1x2 + 11x12 + 22x22 +
Lack-of-fit techniques must be applied in order to check whether
these models are appropriate, since we cannot directly see
whether we are near the optimum (cf. next slides).

department of mathematics and computer science


Models
Far away from
optimum:
first order model

department of mathematics and computer science


Models

Near optimum:
fitting a first
order model
shows lack-offit
(curvature)

department of mathematics and computer science


Models

Near optimum:
second order
model

department of mathematics and computer science

Improvement

In order to efficiently move from current factor settings to

factor setting that yield near-optimal values, 2 methods are
available:
Box/Simplex method
idea: form new full factorials in direction of largest increase
in current full factorial
simple; no statistics needed for implementation
not efficient

Steepest ascent/descent method

idea: use 1st order regression model from fractional factorial
to obtain direction of largest increase (steepest ascent)
perform single runs in direction of largest increase until
increase stops
advanced
recommended since it is more efficient

department of mathematics and computer science

Box method
direction of
largest increase

40.6

41.9

41.2

41.8

41.3

direction of
largest increase
40.0
39.3

41.5
40.9

stop if one has

to return to
previous
settings

department of mathematics and computer science

10

Steepest ascent method

perpendicular
to contour line

direction of
steepest ascent

contour lines of
first-order model

region where
1eorder-model
has been determined

department of mathematics and computer science

11

Optimization scheme

start

end

screening

accept
stationary point
yes

no
full factorial
+ centre points

1st order
model OK?

no

RSM design
(CCD, ...)

fit 2

nd

order model

stationary
point
optimum?

yes

stationary
point
nearby?

yes

no

single observation
in direction
steepest ascent

go to
stationary point
yes

no

better
observation?

department of mathematics and computer science

12

Practical example

goal: maximise yield of chemical reactor

significant factors obtained after screening
experiment:
reaction time
reaction temperature
current factor setting: time = 35 min. temp = 155 C
current yield: 40 %

department of mathematics and computer science

13

Steepest ascent

22-design with 5 centre points:

time: 30 - 40 min; temp: 150 - 160 C
results: montgomery14-1.sfx
there is no significant interaction
there is no significant lack-of-fit
the regression model is significant
Hence, we are not near the optimum.

department of mathematics and computer science

14


Steepest ascent path
outcome analysis of measurement:
yield = 24.94 + 0.155*time + 0.065*temp
with coding:
x1= (time-35)/5

x2 = (temp-155)/5

yield = 40.44 + 0.775*x1 + 0.325*x2

0.775
0.325
direction path: normal vector

step size: 5 min reaction time (choice of chemical engin

5 0.325

0.42 coded step size temp (= 2.1C
5 0.775
department of mathematics and computer science

15

Steepest ascent path experiments

Further experiments with factor settings of experiment nr. 10.

department of mathematics and computer science

16

Near the optimum

Settings experiment 10:

time = 85 min
temperature = 175 C
A 22 design with 5 centre points is executed.
results: montgomery14-4.sfx

Lack-of-fit indicates curvature. Hence, we now are

probably near the optimum.

department of mathematics and computer science

17

Quadratic models

In order to fit a quadratic model (suitable when we

are near the optimum), we must vary the factors at
3 levels.
A 2p-design with centre points does not suffice,
because then all quadratic factors are confounded.
A 3p-design is possible, but not to be recommended:
number of runs grows fast
uses more runs than necessary to fit quadratic
model.

department of mathematics and computer science

18

Response surface designs

The following designs are widely used for fitting a

quadratic model:
Central Composite Design (uniform precision of effect
estimates)

Box-Behnken Design (almost uniform precision of effect

estimates, but usually fewer runs required than for CCD)

The choice between these models is usually

decided by the availability of these designs for a
given number of runs and number of factors.
Note that there are other suitable designs (usually
available in statistical software that supports DOE).

department of mathematics and computer science

19

Central Composite Design

A CCD consists of 3 parts:

factorial points
centre points
axial points
A CCD is often executed by adding
points to an already performed
2p-design (highly efficient, but beware
of blocking!).

department of mathematics and computer science

20

Rotatability

In a CCD there are 2 possible choices:

number of centre points
location axial points
By choosing the axial points at the locations (,0,,0) etc.
with = (# factorial points) , the design becomes
rotatable, i.e. the precision (variance) of the model depends
on the distance to the origin only. In other words, one has
the same precision for all factor estimates.

department of mathematics and computer science

21

Box-Behnken designs

These are designs that

consists of
combinations from 2p-designs.
Properties:
efficient (few runs)
(almost) rotatable
no corner points of
hypercube
(these are extreme
conditions
which are often hard to set)

department of mathematics and computer science

22

Stationary point

Near the optimum usually a quadratic model

suffices:
k

i 1

i 1

Y 0 i xi ii xi2 i j ij xi x j
How do find the optimum after we correctly
estimated the parameters using a response
surface design (CCD or Box-Behnken)?
The next slides show the tools to derive optimal
settings and the pitfalls that have to be avoided.

department of mathematics and computer science

23

Recap: optimisation in dimension1

necessary condition for extremum: 1st derivative = 0
not sufficient:
point of inflection

extra sufficient condition: 2nd derivative 0

department of mathematics and computer science

24

Zero first derivatives: saddlepoint vs.

maximum

saddle point (unfavourable)

x y
2

maximum (favourable)

x y
2

department of mathematics and computer science

25

Determination of type of optimum

Graphically: make contourplot (if 2
factors)
Analytically:

Y 0 i xi x i j ij xi x j
i 1

i 1

2
ii i

T
T
Y

x
b

x
Bx
matrix notation:
0

Note: B must be chosen

/
2

/
2
11
12
13

as symmetric matrix,

see example:
23 / 2
12 / 2 22

13 / 2 23 / 2

33

department of mathematics and computer science

26

 Stationarity and matrix analysis

stationary point (zero first-order derivatives):

Y
1 1
b 2 Bx 0 x0 B b
x
2
characterisation through eigenvalues of matrix B:

Bx x

all eigenvalues positive: min

all eigenvalues negative: max
eigenvalues different signs: saddle po

(the s are sometimes called parameters of canonical f

department of mathematics and computer science

27

 Stationarity and matrix analysis

In StatGraphics:
augment design
add star points
Please note that additional centre points are added
and a block variable.
We can remove the centre points from the data set
and ignore the block variable in the analysis.

StatGraphics results: montgomery14-6.sfx

department of mathematics and computer science

28

 Stationarity and matrix analysis

Use Matlab to avoid manual computations:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2
Analysis Summary
---------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx
Comment: Montgomery table 14-4
Estimated effects for opbrengst
---------------------------------------------------------------------average
= 79.94
+/- 0.11896
A:tijd
= 1.98994 +/- 0.188092
B:temperatuur = 1.03033 +/- 0.188093
AA
= -2.75247 +/- 0.201705
AB
= 0.5
+/- 0.266003
BB
= -2.00253 +/- 0.20171
---------------------------------------------------------------------Standard errors are based on total error with 7 d.f.

department of mathematics and computer science

29

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2
B =
-1.3762
0.1250

0.1250
-1.0013

department of mathematics and computer science

30

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> eig(B)

department of mathematics and computer science

31

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> eig(B)
ans =
-1.4141
-0.9634

both negative maximum

department of mathematics and computer science

32

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> b = [1.98994 ; 1.03033] /2
Analysis Summary
---------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx
Comment: Montgomery table 14-4
Estimated effects for opbrengst
---------------------------------------------------------------------average
= 79.94
+/- 0.11896
A:tijd
= 1.98994 +/- 0.188092
B:temperatuur = 1.03033 +/- 0.188093
AA
= -2.75247 +/- 0.201705
AB
= 0.5
+/- 0.266003
BB
= -2.00253 +/- 0.20171
---------------------------------------------------------------------Standard errors are based on total error with 7 d.f.
33

department of mathematics and computer science

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> b = [1.98994 ; 1.03033] /2
b =
0.9950
0.5152

department of mathematics and computer science

34

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b

department of mathematics and computer science

35

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b
spcoded =
0.3893
0.3059

< 1.414 (distance star point)

inside experimental region

department of mathematics and computer science

36

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]

time 85
temp 175
x1
, x2
5
5
time 5 x1 85, temp 5 x2 175

department of mathematics and computer science

37

 Stationarity and matrix analysis

In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]
sporiginal =
86.9463
176.5293

department of mathematics and computer science

38

Optimization scheme

start

end

screening

accept
stationary point
yes

no
full factorial
+ centre points

1st order
model OK?

no

RSM design
(CCD, ...)

fit 2

nd

order model

stationary
point
optimum?

yes

stationary
point
nearby?

yes

no

single observation
in direction
steepest ascent

go to
stationary point
yes

no

better
observation?

department of mathematics and computer science

39

Multiple responses

te m p e r a tu u r

If more than 1 response variable needs to be

optimised, then a graphical way of optimising may
be achieved by overlaying contour plots in case
there are only 2 independent variables.
Overlay Plot
180
molecular_weight
opbrengst
viscositeit

178
176
174
172
170
80

82

84

86

88

90

tijd

department of mathematics and computer science

40

Evolutionary Operation (EVOP)

Optimisation of a running production process is not

always possible or may not be allowed because of costs:
involves interruption
may (temporarily) yield low quality products
An alternative is Evolutionary Operation:
experimentation within running operation
frequent execution of 2k-designs, starting at current
settings
high and low setting of factors are close to each other,
thus no risk of low quality products

department of mathematics and computer science

41

Software

StatLab optimisation:
http://www.win.tue.nl/statlab
Interactive software for teaching DOE through cases
Box: http://www.win.tue.nl/~marko/box/box.html
Game-like demonstration of Box method
Matlab virtual reactor: Statistics toolbox -> Demos ->
Empirical Modeling -> RSM demo
Statgraphics:
menu choice Special -> Experimental Design
design experiment with pre-defined catalogue
analysis of experiments with ANOVA

department of mathematics and computer science

42

Literature

J. Trygg and S. Wold. Introduction to Experimental

Design What is it? Why and Where is it Useful?,
Homepage of Chemometrics, editorial August 2002:
http://www.acc.umu.se/~tnkjtg/Chemometrics/editori
al/aug2002.html
DOE booklet from Umetrics: http://www.umetrics.com
/pdfs/books/DOEbook.pdf
Introduction to DOE from moresteam.com http://
www.moresteam.com/toolbox/t408.cfm
StatSoft Electronic Statistics Textbook, chapter on
experimental design
NIST Engineering Statistics Handbook:
http://www.itl.nist.gov/div898/handbook/

department of mathematics and computer science

43