Class Notes PDF

EXPERIMENTAL DESIGN STRATEGIES
A Short Course for Chemical Engineering Graduate Students

Massachusetts Institute of Technology
21 - 23 March, 2005
y = b0 + x ′b + x ′Bx
RESPONSE
75
65
55
5
4
45
55 3 X1
45
X2 35
25 2
INSTRUCTORS:
Debashis Neogi, Ph.D. - Lead Systems Engineer

Sanjay Mehta, Ph.D. - Senior Principal Systems Engineer
Computational Modeling Center

© Air Products & Chemicals, Inc.
Allentown, PA 18195
INDEX
SECTION I: Introduction
SECTION II: Regression Analysis
SECTION III: Factorial and Screening Design Strategies
SECTION IV: Response Surface Concepts
SECTION V: Mechanistic Model Building
SECTION VI: Mixture Design

EXPERIMENTAL DESIGN STRATEGIES
About This Course...
Every data collection either explicitly or implicitly reflects the choice of an experimental design
strategy. This choice of design strategy, more than any other factor, determines how successful
an experimental program will be. The impact of using poor design strategies include technical
stagnation, inefficient use of resources, incorrect conclusions, and incomplete or missed
solutions.
This short seminar examines the technical considerations involved in the selection of an
experimental design strategy. Well selected design strategies are powerful tools for the
systematic study of processes (laboratory or plant). This material represents a core set of
knowledge which has proven to be applicable to many problems at Air Products and Chemicals,
Inc.
The primary focus of the seminar is to develop an understanding of the basics of

experimental design, i.e., how to properly select a design strategy and lay out a set of
experiments/tests. The goal of experimental design is to get the most information possible from
a given number of experiments. Data analysis and interpretation issues are covered, but not in as
much detail as design issues.
This seminar was originally developed by T. Bzik, S. Mehta, D. Neogi, J. Sheesley, and C. A.
Valenzuela from the Research and Engineering Systems Department. The notes are reviewed
and modified every year to incorporate changes in the discipline.
The instructors will appreciate suggestions on how to improve the course.

BIBLIOGRAPHY
1. Barker, Thomas B. (1985), Quality By Experimental Design, Marcel Dekker Inc.
2. Box, George E.P. and Behnken, D.W. (1960), "Some New Three Level Designs for the Study of
Quantitative Variables," Technometrics, 2, pp 455-475.
3. Box, George E.P. and Draper, Norman R. (1987), Empirical Model-Building and Response
Surfaces, John Wiley and Sons, Inc., NY.
4. Box, George E.P.; Hunter, William G.; and Hunter, J. Stuart (1978), Statistics for Experimenters,
John Wiley and Sons, Inc., NY.
5. Cornell, John A. (1990), Experiments with Mixtures, 2nd Edition, John Wiley and Sons, New
York.
6. Draper, Norman R. and Smith, H. (1998). Applied Regression Analysis, 3rd Edition, John
Wiley and Sons, Inc., NY.
7. Montgomery, Douglas C. (1997), Design and Analysis of Experiments, John Wiley and Sons,
4th Edition.
8. Myers, Raymond H. and Montgomery, D. C. (1995), Response Surface Methodology, Virginia

Polytechnic Institute.
9. Neter, J., Wasserman W., and Kutner M. H. (1985). Applied Linear Statistical Models, Richard
D. Irwin, Inc. Illinois.
10. Taguchi,G.and Konishi,S.(1987), Taguchi Methods: Orthogonal Arrays and Linear Graphs,
American Supplier Institute, Inc.
11. Schmidt, S. R. and Launsby, R. G. (1992), Understanding Industrial Designed Experiments, Air
Academy Press.
12. Wonnacott, T. H., Wonnacott, R. J. (1981), Regression: A Second Course in Statistics, Robert E.
Krieger Publishing Co.
Frequently conclusions are easily drawn from a well designed
experiment, even when rather elementary methods of analysis
are employed. Conversely, even the most sophisticated analyses
cannot salvage a badly designed experiment.
Box, Hunter, and Hunter
Statistical methods are aids to engineering judgment and not a

substitute for it.
Harold F. Dodge
Always graph your data in some simple way - always.!!

Ellis R. Ott
SECTION I
INTRODUCTION
Introduction Intro-2
Table of Contents
WHAT IS DESIGN OF EXPERIMENTS? ...................................................................................................... 3
A TYPICAL STRATEGY ...................................................................................................................................... 4
ABOUT THIS COURSE... .................................................................................................................................... 5
ONE-AT-A-TIME..................................................................................................................................................... 6
THE DATA............................................................................................................................................................. 7
REGRESSION ANALYSIS ................................................................................................................................ 8
A BOX-BEHNKEN DESIGN... ................................................................................................................................ 11
The best (parsimonious) model... ......................................................................................................... 12
________________________________
Computational Modeling Center, 2005
©Air Products and Chemicals, Inc.
WHAT IS DESIGN OF EXPERIMENTS?
Main objective of most experiments -- assess the

effects of certain factors on some response variables.
e.g., estimate the effects of temperature and pressure on

yield of a certain chemical.
Typically, we assume a relationship between

• A response, y, and
• Factors (also called the independent variables), x1,
x2,...,xn, which can be expressed as
y = φ( x 1 , x 2 , . . . , x n )
The function φ represents an approximation to the true

underlying relationship.
Mechanistic models based on first principles may be

used (yes, this can be very difficult some times!).
The challenge is to find this relationship with a

minimum number of experiments. Furthermore, we
would like the estimate of the function φ to be such that
the effects of each xi is clearly quantified.
________________________________
A Typical Strategy
While there are various ways to analyse the results of
experiments, this course will emphasize REGRESSION
ANALYSIS as the principal tool of choice.
Objectives Experimental Design

Strategy
data
structure
Data Collection Raw Data
Input
Data
Models
y = f ( x1 , x2 , . . . , x n )
regression
Analysis
selection
Best
Model
________________________________
ABOUT THIS COURSE...
This course will emphasize four major topics:
• Regression analysis
• Experimental design strategies for empirical

models
• Response Surface Methodology
• Mechanistic Model Building
________________________________
ONE-AT-A-TIME
A computer generated example: 3 variables and one
response:
x1 , x2 , x3 y
y = f ( x1 , x2 , x3 )
A one-at-a-time approach: vary one factor while holding

the other two constant.
________________________________
THE DATA
Obs # X1 X2 X3 Y
1 1 3 3 38.99
2 2 3 3 66.53
3 3 3 3 92.94
4 4 3 3 122.34
5 5 3 3 147.62
6 3 1 3 40.25
7 3 2 3 68.02
8 3 3 3 94.18
9 3 4 3 119.48
10 3 5 3 146.48
11 3 3 1 95.26
12 3 3 2 94.90
13 3 3 3 94.04
14 3 3 4 93.58
15 3 3 5 91.36
Let's see how good the strategy is...
________________________________
REGRESSION ANALYSIS
We need measures of performance to evaluate the

model(s) obtained.
Two such measures are:
Measure Meaning Criterion
R2 - Coefficient of A measure of the 0 ≤ R 2 ≤ 1,

determination variation explained by the higher the better
the estimated regression
equation
RMSE - Mean Unbiased estimate of σ2 The smaller the better

square error
On the surface, this appears to be a good model...
R2 = 0.9997
RMSE = 0.7334
The model is:
y = −68 . 61 + 55 . 67 x 1 − 9 . 13 x 1 x 3 + 9 . 17 x 2 x 3 − 0 . 16 x 12 − 0 . 19 x 22 − 0 . 17 x 32
(Note, however, that when we tried the full quadratic

model, the ( x ′x) matrix was singular!)
−1
________________________________
The model performs well for the chosen points...

One-at-a-Time Example
Quadratic Fit - Best Model
Dep Var Predict Std Err Lower95% Upper95%

Obs Y Value Predict Predict Predict Residual
1 38.9900 38.8207 0.686 36.5045 41.1370 0.1693

2 66.5300 66.6091 0.336 64.7486 68.4696 -0.0791
3 122.3 121.2 0.336 119.4 123.1 1.1169
4 147.6 148.0 0.686 145.7 150.4 -0.4287
5 40.2500 40.5460 0.686 38.2297 42.8623 -0.2960
6 68.0200 67.4979 0.336 65.6375 69.3584 0.5221
7 119.5 120.3 0.336 118.4 122.1 -0.8019
8 146.5 146.1 0.686 143.8 148.4 0.3660
9 95.2600 95.2175 0.686 92.9013 97.5338 0.0425
10 94.9000 94.8178 0.336 92.9573 96.6783 0.0822
11 93.5800 92.9938 0.336 91.1333 94.8543 0.5862
12 91.3600 91.5695 0.686 89.2533 93.8858 -0.2095
13 92.9400 94.0766 0.295 92.2535 95.8996 -1.1366
14 94.1800 94.0766 0.295 92.2535 95.8996 0.1034
15 94.0400 94.0766 0.295 92.2535 95.8996 -0.0366
Let's see how well it predicts...
X1 X2 X3 Pred Y True Y
5 5 5 197.74 230.00
1 2 3 12.24 30.00
1 1 1 -13.42 22.00
3 3 3 94.08 94.00
Very poor data structure!!!
________________________________
Conclusions:
• Approach didn't permit estimation of the true model
• Made an incomplete and incorrect model appear to be

acceptable
• Regression model produced biased estimates (at best!);

some computer packages would not even attempt to
solve the problem. SAS can use generalized inverses.
In general, if you use a one-at-a-time approach be

prepared to experience:
• Partial confounding among the predictors - won't be

able to clearly identify effects!
• Time and energy wasted in dealing with data errors,

inconsistent data collection procedures
• Unreliable models
________________________________
A Box-Behnken design...
(same number of experiments)
• We could have used a factorial design which requires

only 11 points, but...
• Let's assume we have the luxury of 15 observations...
Obs # X1 X2 X3 Y
1 5 5 3 231.45
2 5 1 3 64.13
3 1 5 3 59.77
4 1 1 3 21.80
5 5 3 5 144.49
6 5 3 1 150.48
7 1 3 5 38.47
8 1 3 1 40.76
9 3 5 5 142.04
10 3 5 1 149.69
11 3 1 5 39.27
12 3 1 1 42.38
13 3 3 3 94.60
14 3 3 3 94.70
15 3 3 3 95.58
________________________________
The best (parsimonious) model...

Results
Response Model : Y1 (Y1)
Number of terms in the User Selected Full Model : 10

Model Terms : Intercept, X1, X2, X3, X1X2, X1X3, X2X3, X1X1, X2X2, X3X3
Weights : Equal weights
Final/Best Subset Model

Model Parameter Estimates
95% Confidence Interval

Model Coefficient Standard
Labels
Terms Estimate Error Lower Upper
Bound Bound
Intercept 11.8434 2.0215 6.5192 17.1675
X1 X1 2.6063 0.5681 1.1099 4.1026
X2 X2 1.7075 0.5681 0.2112 3.2038
X3 X3 -1.19 0.2423 -1.828 -0.552
X1X2 X1*X2 8.0844 0.1713 7.6332 8.5355
ANOVA (Analysis of Variance)
Sum of Degrees of Mean Conclusio

Model Terms F-value P-value
Squares Freedom Square n
Model 48880.3333 4 12220.0833 6507.2057 5.0108E-17 Significant
X1 39.5202 1 39.5202 21.0446 0.001 Significant
X2 16.9632 1 16.9632 9.0329 0.0132 Significant
X3 45.3152 1 45.3152 24.1304 0.0006 Significant
X1X2 4182.8556 1 4182.8556 2227.3745 4.3976E-13 Significant
Residuals 18.7793 10 1.8779
Lack of Fit 18.1977 8 2.2747 7.8223 0.1182 No LOF
Pure Error 0.5816 2 0.2908
Total 48899.1126 14
________________________________
Coefficient of Determination R^2 : 0.9996

Standard Error of Estimate (Root Mean Square Error) : 1.3704
Model Prediction and Residual Analysis
Standardized
Ob Observed Predicted Residuals
Residuals Leverage Cook's D Atkinson's T
s Response Response
Internal External
1 231.45 231.9515 -0.5015 0.5667 -0.5559 -0.5357 0.0808 -0.8664
2 64.13 63.434 0.696 0.5667 0.7715 0.7548 0.1557 1.2206
3 59.77 59.839 -0.069 0.5667 -0.0765 -0.0726 0.0015 -0.1174
4 21.8 20.6715 1.1285 0.5667 1.251 1.2922 0.4093 2.0898
5 144.49 145.3128 -0.8227 0.3167 -0.7263 -0.7079 0.0489 -0.6816
6 150.48 150.0728 0.4073 0.3167 0.3595 0.3433 0.012 0.3305
7 38.47 37.8753 0.5948 0.3167 0.525 0.5051 0.0255 0.4863
8 40.76 42.6353 -1.8752 0.3167 -1.6554 -1.8432 0.254 -1.7745
9 142.04 143.5153 -1.4752 0.3167 -1.3023 -1.3558 0.1572 -1.3052
10 149.69 148.2753 1.4148 0.3167 1.2489 1.2896 0.1446 1.2416
11 39.27 39.6728 -0.4028 0.3167 -0.3555 -0.3394 0.0117 -0.3268
12 42.38 44.4328 -2.0527 0.3167 -1.8121 -2.0977 0.3043 -2.0195
13 94.6 93.974 0.626 0.0667 0.4728 0.4537 0.0032 0.1715
14 94.7 93.974 0.726 0.0667 0.5484 0.5282 0.0043 0.1997
15 95.58 93.974 1.606 0.0667 1.2131 1.2462 0.021 0.471
Shapiro-Wilks W : 0.9381
Significance level: 0.5923
________________________________
Let's see how well it predicts at the same points as

before...
Model Prediction Results
Project : Introduction
Analysis : RS Model 2
Prediction : Model Prediction 1
Response : Y1 (Y1)
Prediction Points Predicted 95% Confidence Limits 95% Prediction Limits

#
Values
X1 X2 X3 Lower Upper Lower Upper
1 5 5 5 229.5715 227.0321 232.1109 225.6001 233.5429
2 1 2 3 30.4634 28.8321 32.0946 27.0016 33.9252
3 1 1 1 23.0515 20.5121 25.5909 19.0801 27.0229
4 3 3 3 93.974 93.1856 94.7624 90.8205 97.1275
X1 X2 X3 Pred Y True Y Residuals
5 5 5 229.5715 230 -0.429

1 2 3 30.4634 30 -1.07
1 1 1 23.0515 22 1.53
3 3 3 93.974 94 -0.58
Incidentally, the true model was...
y = 10 + 3x1 + 2 x2 − x3 + 8 x1 x2 + ε
where the errors are normally distributed, i.e.,
________________________________
ε = N ( 0,1)
Stat-Studio predicted the following model:
y =11.8434 + 2.6063*X1 + 1.7075*X2 - 1.19*X3 + 8.0844*X1*X2
Note that the 2nd approach yielded a model very similar

to the correct true model.
Which approach provides a more robust result?
________________________________
SECTION II
REGRESSION ANALYSIS
Regression Analysis REG - 1
Table of Contents
BASIC ASSUMPTIONS........................................................................................................................................3
NOMENCLATURE.................................................................................................................................................4
WHY IS IT CALLED 'LINEAR REGRESSION?' .........................................................................................5

LEAST SQUARES METHOD.............................................................................................................................6
THE CASE OF N VARIABLES ................................................................................................................................8
THE IMPORTANCE OF ( X ′X ) −1 ...............................................................................................................11

MODEL EVALUATION......................................................................................................................................16
RESIDUAL ANALYSIS ......................................................................................................................................36
MULTICOLLINEARITY ....................................................................................................................................46
VARIANCE INFLATION FACTORS ..............................................................................................................48
EIGENVALUES AND CONDITION NUMBERS........................................................................................49
POSSIBLE REMEDIAL ACTIONS .................................................................................................................51
THE CORRELATION COEFFICIENT ...........................................................................................................52
M O D E L B U I L D I N G ........................................................................................................................................55
INPUTTING THE DATA IN SAS....................................................................................................................60

FINDING THE ‘BEST’ MODEL.......................................................................................................................61
USEFUL TRANSFORMATIONS ....................................................................................................................61
TRANSFORMATIONS OF THE INDEPENDENT VARIABLES ........................................................63
2 LEVEL VARIABLE: ...............................................................................................................................................64
POLYNOMIAL EXPANSIONS.........................................................................................................................71
ANALYZING A TYPICAL COMPUTER REPORT....................................................................................74

I. SAS OUTPUT ..................................................................................................................................................75
II. THE LANGUAGE OF REGRESSION .....................................................................................................76
III. DETAILED EXAMPLE: REGRESSION ANALYSIS IN SAS..........................................................78
IV. MORE EXAMPLES: LINEAR REGRESSION ANALYSIS IN SAS ...............................................80

Basic concepts
True Model: y = β 0 + β1 x1 +... +β n x n + ε
POPULATION
SAMPLE
Best estimate: y = b0 + b1 x1 +...+ bn x n

BASIC ASSUMPTIONS
We assume there is a ‘true’ model (population):
y = β0 + β1 x1 + β2 x 2 +...+ βn x n + ε
We estimate β0 , β1 ,... βn using b0 , b1 ,..., bn
Estimated Model:
y$ = b0 + b1 x + b2 x 2 +...+ bn x n
Notice that E(ε)=0
We need to find a method to estimate the coefficients

based on some ‘goodness’ criteria.

NOMENCLATURE
INDEPENDENT VARIABLES
(deterministic or random variables
with constant variance)
y = β 0 + β1 x1 +... + β n x n + ε
COEFFICIENTS
(Random variables)
σ 2b
β ERROR TERM
DEPENDENT (random variable)
VARIABLE OR
σ 2ε
RESPONSE
(Random variable)
σ 2y
0
E(y)

WHY IS IT CALLED 'LINEAR REGRESSION?'
It is important to note that linear regression requires also that the

resulting models be linear in the coefficients!
For example, we may have a kinetics equation of the type
⎛ E ⎞
−⎜ ⎟ m
W = K0e ⎝ RT ⎠
[
∏ iA ] p i
which becomes
m
+ ∑ pi ln[ Ai ]
E
ln W = ln K 0 −
RT 1
Under certain conditions, this equation is linear with respect to the

coefficients!
WHAT ABOUT:
− bx
• y = ae
a + bx
• y =
c + dx
θ − θ 2t −θ1t
• Y = 1
e − e +ε
θ1 − θ 2

LEAST SQUARES METHOD
The objective is to find the estimates of β such that the sum of the
square errors is minimized. This is known as the Least Squares
Method (LSE).
Minimize Q = ∑e 2
i = ∑ ( y i − y$ i ) 2
In the case of one independent variable the solution is quite simple:
Min Q = ∑ ( y − b0 − b1 x1 )
2
The minimum is obtained by taking the first derivatives
∂Q
=0 ⇒ nb 0 + b1 ∑ x i = ∑y
∂b0
i
i i
∂Q
=0 ⇒ b0 ∑ x i + b1 ∑ x i2 = ∑x y
∂b1
i i
i i i
This system of equations is also known as the Normal System of

equations.

Solving for b0 , b1
∑ ( x − x )( y − y )
i i
b1 = i
∑( x − x )i
i
2
b0 = y − b1 x
the OLS equation becomes
y$ i = b0 + b1 x i

THE CASE OF N VARIABLES
Matrix Notation
In matrix notation the equations become
Population Sample Model

Model
y = Xβ + ε ====> y$ = Xb
Where the dimensions are
y: nx1
X: nx(p+1)
b: (p+1)x1
ε : nx1
Notice the column of 1’s in the X matrix, representing a ‘dummy’

variable x0 = 1 in the model.
⎡1 x11 x12 ... x1 p ⎤

⎢1 x x 22 ... x 2 p ⎥
X=⎢ ⎥
21
⎢. . . . ⎥
⎢ ⎥
⎣1 x n1 xn2 ... x np ⎦

The least squares method then is formulated as follows after

substituting b for β:
Minimize Q = ( Y − Xb)′ ( Y − Xb)
Q = Y ′Y − b′ X ′Y − Y ′Xb + b′ X ′Xb
which, after some manipulations becomes
Q = Y ′Y − 2b′ X ′Y + b′ X ′Xb
Taking derivatives with respect to b and equating to zero:
∂Q
= −2 X ′Y + 2 X ′Xb = 0
∂b
which yields the normal equations
( X ′X )b = X ′y
whose solution is
b = ( X ′X ) − 1 X ′y

The coefficients follow a Normal distribution with
σ b2
Mean = E ( b1 ) = β1
Variance = σ b2 = σ 2 ( X ′X ) −1 or, approximately,
s 2 ( b ) = MSE( X ′X )−1

THE IMPORTANCE OF ( X ′X )
−1
1. The matrix X ′X is called the variance-covariance matrix.

−1
The inverse, ( X ′X ) , is called the information matrix.
2. If we choose the settings for the variables in the matrix X
carefully, we could make the matrix X ′X orthogonal of rank

(p+1). For example, for p = 3 variables,
⎡1 x11 x12 x13 ⎤

⎡1 1 ... 1 ⎤ ⎢1 x21 x 22 x 23 ⎥
⎢ ⎥
⎢x x21 ... ⎥
x n1 ⎢1 x31 x32 x33 ⎥
X ′X = ⎢ ⎥⎢
11
⎥
⎢ x12 x22 ... xn2 ⎥⎢. . . . ⎥
⎢ ⎥
⎣ x13 x23 ... x n3 ⎦ ⎢ . . . . ⎥
⎢ ⎥
⎣1 x n1 xn2 x n3 ⎦
which, under conditions of orthogonality, yields
⎡n 0 0 0 ⎤
⎢0 0 ⎥
X ′X = ⎢
∑x 2
i1 0
⎥
⎢0 0 ∑ xi22 0 ⎥
⎢
⎣0 0 0 ∑ xi23 ⎥⎦

Obtaining the inverse of an orthogonal matrix is trivial!

Choosing the right settings for the x variables is the
principle behind design of experiments.
−1
3. The matrix ( X ′X ) is part of the equation for the variances
of b, Y, etc. Hence, it is important to structure the data
collection procedure to compute it accurately.
Multicollinearity is a typical problem, leading to a
determinant which is close to zero.

A FEW DETAILS...
As indicated before, the response vector y is a random variable. For

instance, in the case of one variable,
σ 2
σ 2
σ 2
Notes:
1. The y distributions have the same shape at each x value.

2. The variance of ε, denoted by σ2, is the same for all values of
x.
2
3. The variance of y equals σ and is the same for all values of
x1,x2,...,xn.

Properties and Assumptions in Regression

Analysis for the Model y = Xβ + ε
Properties: The regression residuals, ei = y i − y$ i , have the following

properties.
1. The mean value of the residuals ei is zero ===> E( ei ) = 0
2. The residuals ei are uncorrelated with the predicted yi, i.e.,

∑ ei yi = 0
3. The residuals ei are uncorrelated with xi. In other words,
∑ ei xi = 0 .
Assumptions regarding ε:
1. The disturbances or error terms (also called residuals), ε i , follow a
normal distribution N( 0, σ )
2
2. E( ε i ) = 0
Variance = σ2
-3σ’ -2σ’ -σ’ 0 +σ’ +2σ’ +3σ’

3. There is no autocorrelation between the ε i 's, that is,

E{[ε i − E ( ε i )]′[ε j − E ( ε j )]} = 0
4. Homoscedasticity or equal variance of ε i : the variance of the error

term is some positive constant σ 2 . This implies that the y
populations corresponding to values of x have the same variance,
σ2.
5. The error term ε i and the explanatory variable xi are

uncorrelated.

MODEL EVALUATION
Typical questions:
1. IS THE MODEL SIGNIFICANT?
2. ARE THE COEFFICIENTS

SIGNIFICANT?
3. HOW GOOD IS THE MODEL?
To answer these questions we need an estimate for σ2.
We’ll use information from the Analysis of Variance

(ANOVA) table to obtain this estimate.

ANOVA TABLE
ei = yi − yˆ i
y
yˆ i − y i
x x
The following equality holds
y i − y = ( y i − y$ i ) + ( y$ i − y )

ANOVA TABLE
The ANOVA table is based on the decomposition of the sum of squares

for the total variation:
∑ (y
i
i − y ) 2 = ∑ ( y i − y$ i ) 2 + ∑ ( y$ i − y ) 2
i i
SST = SSE + SSR
Here SST represents the total sum of squares; SSE is the sum of
squares due to error and SSR is the sum of squares due to regression.
Typically, this is shown as a table.
Mean
SOURCE SS df Squares F
Regression ∑ ( y$i − y )2 p MSR =

SSR
F =
MSR
MSE
(Model) i p
Error ∑ ( yi − y$i )2 n-p-1 MSE =

SSE
i n − p−1
TOTAL ∑ ( yi − yi )2 n-1
i
Best estimate
of σ
2
Where: p = Number of coefficients (excluding b0)

n = Number of data points

Using the ANOVA table in conjunction with other information we can

answer the following questions regarding the regression model:
1. Is the model significant?

===> Use F tests and associated p-values
2. Are the coefficients significant?

===> Use t-tests and associated p-values
3. How good is the model?

===> typically, use R2 and RMSE
We now have an estimate for σ 2 ...
SSE
Estimate of σ = MSE = 2
n − p −1

Problem: Gloss Properties
Description: A study to investigate the gloss of a certain coating

exposed to cool ambient temperatures (0-10 0C ) has been undertaken.
The two factors studied were the outside temperature and the curing
time.
Data: Data are shown below. Gloss is shown as a scaled number.
X1 X2 Y
(Temp, 0 C ) (Time, min.) (Gloss)
4.000 78.000 24.000

7.000 100.000 43.000
1.000 86.000 23.700
5.000 82.000 34.300
8.000 86.000 35.800
10.000 84.000 38.000
.000 75.000 22.200
1.000 80.000 23.100
6.000 83.000 30.000
6.000 91.000 33.000
9.000 88.000 38.000
2.000 73.000 26.600
10.000 75.000 36.200
5.000 81.000 31.600
6.000 74.000 29.000
8.000 87.000 34.000
4.000 79.000 30.100
6.000 94.000 33.900
3.000 70.000 28.200
3.000 89.000 30.000

PLOTS
Project Name : MIT One at a TIME Project Name : MIT One at a TIME
Plot Name : User Plot1 [RS Model 2] Plot Name : User Plot1 [RS Model 2]
3/10/2004 10:54:13 AM 3/10/2004 10:58:19 AM
45 45
40 40
Observed Gloss(Y1)
Observed Gloss(Y1)
35 35
30 30
25
25
0 1 2 3 4 5 6 7 8 9 10 11
70 80 90 100
Temp(X1)
Time(X2)
3D Graph 1
45
40
35
Gloss
30
12
25 10
8
6
p
m
20 4
100
Te
95 2
90
85
80 0
75
70
Tim e 65

Stat-Studio: Output
Data Structure Evaluation
Project : Gloss Properties of Coating
Independent variables bounds
Continuous variables
Mi
Variable Label Max
n
X1 Temp (C) 0 10
X2 Time (Min) 70 100
Response Model : Gloss (Y1)

Model Terms : Intercept, X1, X2
For achieving reliable modeling results, it is essential that the experimental data has a proper robust
structure.This structure should be such that it supports an unbiased estimation of the quadratic model
used in response surface methodology.
The data structure was investigated through a Variance Inflation Factor (VIF) analysis.
Results:
VIF for each term in the model
Intercept = 0
X1 X2
Temp (C) Time (Min)
1.1268 1.1268
Conclusion:
Data has the (near) orthogonal structure required for the robust modeling.
Second order polynomial modeling should be robust.
Design is balanced.
Correlation Between the Model Terms:

Following Matrix shows the Pearson Pair-wise Coefficients for the each pair of terms in the quadratic
model.

Main-effects:
X1 X2
X1 1 0.3355
X2 0.3355 1
3/18/2004 2:11:11 PM
Results
Response Model : Gloss (Y1)

Model Terms : Intercept, X1, X2


Labels
Bound Bound
Intercept 3.1739 6.1561 -11.958 18.3059
X1 Temp (C) 1.4039 0.1986 0.9158 1.892
X2 Time (Min) 0.2509 0.0774 0.0607 0.441
Sum of Degrees of Mean

Model Terms F-value P-value Conclusion
Squares Freedom Square
X1 292.4465 1 292.4465 49.9874 1.8806E-06 Significant
X2 61.5418 1 61.5418 10.5192 0.0048 Significant
Residuals 99.457 17 5.8504
Lack of Fit 99.457 17 5.8504 0 0 LOF can not be tested
Pure Error 0 0 0
Total 599.7855 19


Observed Predicted Standardized Residuals

Obs Residuals Leverage Cook's D Atkinson's T
Response Response
Internal External
1 24 28.3586 -4.3586 0.0727 -1.8713 -2.0374 0.0916 -1.3584
2 43 38.0898 4.9102 0.3215 2.4645 2.9823 0.9592 4.8865
3 23.7 26.154 -2.454 0.2037 -1.137 -1.1475 0.1103 -1.3817
4 34.3 30.7661 3.5339 0.0506 1.4995 1.5616 0.0399 0.858
5 35.8 35.9813 -0.1813 0.0976 -0.0789 -0.0766 0.0002 -0.0599
6 38 38.2873 -0.2873 0.1963 -0.1325 -0.1286 0.0014 -0.1513
7 22.2 21.9903 0.2097 0.2227 0.0983 0.0954 0.0009 0.1216
8 23.1 24.6487 -1.5487 0.1563 -0.6971 -0.6861 0.03 -0.7029
9 30 32.4208 -2.4208 0.054 -1.029 -1.0309 0.0202 -0.5865
10 33 34.4279 -1.4279 0.1123 -0.6266 -0.615 0.0166 -0.5207
11 38 37.887 0.113 0.1404 0.0504 0.0489 0.0001 0.047
12 26.6 24.2964 2.3036 0.1613 1.0399 1.0426 0.0693 1.0883
13 36.2 36.0294 0.1706 0.3322 0.0863 0.0838 0.0012 0.1407
14 31.6 30.5152 1.0848 0.0528 0.4608 0.4499 0.0039 0.2528
15 29 30.1629 -1.1629 0.145 -0.5199 -0.5085 0.0153 -0.4984
16 34 36.2322 -2.2322 0.1003 -0.973 -0.9714 0.0352 -0.7722
17 30.1 28.6095 1.4905 0.0662 0.6377 0.6262 0.0096 0.3968
18 33.9 35.1806 -1.2806 0.1679 -0.5804 -0.5687 0.0227 -0.6082
19 28.2 24.9476 3.2524 0.1995 1.5029 1.5657 0.1876 1.8605
20 30 29.7144 0.2856 0.1468 0.1278 0.1241 0.0009 0.1225
Parity Plot

Project Name : Gloss Properties of Coating

Plot Name : Parity Plot for Gloss(Y1) [RS Model 1]
3/18/2004 2:15:24 PM
44
42
40
38
36
Observed Response
34
32
30
28
26
24
22
20
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
Predicted Response
Variable Importance Plot

Plot Name : Variable Importance Plot for Gloss(Y1) [RS Model 1]
3/18/2004 2:15:58 PM
Tem p (C) 0.7412

Model Terms
Tim e (Min) 0.34
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Standardized Estim ates

1. IS THE MODEL SIGNIFICANT?
The F statistic (see ANOVA table) is used to test the hypothesis:
H 0 : β 1 = ... = β i = 0
H α : At least one β i ≠ 0
The rejection criterion is
Reject H 0 if the p - value < α

From the computer output,

X1 292.4465 1 292.4465 49.9874 1.8806E-06 Significant
X2 61.5418 1 61.5418 10.5192 0.0048 Significant
Residuals 99.457 17 5.8504
Pure Error 0 0 0
Total 599.7855 19
The level of significance, α, is set at a conservative value of α= 0.05.
CONCLUSION:
=====> Since 2.32e-7 < 0.05, H0 is rejected.

Model is Significant.

2. ARE THE COEFFICIENTS SIGNIFICANT?
H0: βi = 0
Hα : βi ≠ 0
From the ANOVA table,
Decision rule:
Reject H 0 if the p - value < α

The observed values are:
Parameter: X1 p-value= 1.8806E-06

Parameter: X2 p-value=0.0048.
CONCLUSION:
======> Both coefficients are significant!

3. HOW GOOD IS THE MODEL?
Typically, we rely on these criteria to assess how good a model is: the
best model will have
1. The smallest estimate of error, i.e., RMSE = MSE

2. The largest coefficient of determination, R2 .
SSR
where, R 2
= can be obtained from the ANOVA table,
SST
500.329
R2 = = 0.834
599.786
RMSE = 2.4188
With an R2=0.834, the model appears to be good. This, however, is a

judgment call. In some cases, a model with a low R2 may be the best
you can get...

USING THE REGRESSION EQUATION FOR

ESTIMATION AND PREDICTION
Once the regression equation is obtained, it can be used to estimate the

mean value of y for a given x or as a predictive tool.
a) The mean value of gloss (y) for all coatings rated at 5 0C and a curing
time of 90 minutes is:
yˆ = 3.174 + 1.404 x1 + 0.251x 2 = 32.77
b) The predicted value of y for a particular coating rated at 5 0C and a

curing time of 90 minutes is:
yˆ = 3.174 + 1.404 x1 + 0.251x 2 = 32.77
As before, confidence intervals can be defined for both

an individual result as well as a mean result.

CONFIDENCE INTERVALS
1. The 95% confidence interval of the gloss for one particular coating
is
yˆ c ± tα / 2,n− p −1 MSE[1 + X′c ( X′X) −1 X c ]

= yˆ c ± t 0.025,17 ⋅ s ( yˆ c )
= 32.7731 ± 2.11 5.85 * [1.106676]
= 32.7731 ± 5.3687
or, Prob ( 27.40 ≤ yˆ c ≤ 38.14) = 0.95
2. The 95% confidence interval of the mean gloss for all coatings
rated at 5 0C and a curing time of 90 minutes is
yˆ m ± tα / 2,n − p −1 MSE[ X′m ( X′X) −1 X m ]

= yˆ m ± t 0.025,17 ⋅ s ( yˆ m )
= 32.7731 ± 2.11 5.85 * [0.106676]
= 32.7731 ± 1.6668
or, Prob (31.11 ≤ yˆ m ≤ 34.44) = 0.95
The confidence interval for the individual prediction

or estimation is wider than the corresponding interval
for the mean value.


Model Prediction Results
Prediction : Model Prediction 1
Response : Gloss (Y1)
Prediction Points 95% Confidence Limits 95% Prediction Limits

Predicted
#
Time Values
Temp (C)(X1) Lower Upper Lower Upper
(Min)(X2)
1 5 90 32.7731 31.1071 34.4392 27.4049 38.1414

A Glance at the Residuals...
A basic first step in evaluating the adequacy of the model is to look at

the residuals.
The plot of Residuals versus Estimates ( e vs y$ ) should reveal no

patterns and the residuals should be randomly scattered about the
center line.

Plot Name : Residual Plot for Gloss(Y1) [RS Model 1]
3/18/2004 2:36:04 PM
1
Standardized Residuals
-1
-2
-3
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Predicted Response
No patterns appear to exist.

Prediction Plots
Plot Name : User Plot1
10/3/2002 2:59:56 PM
Predicted Gloss(Y1)
Observed Gloss(Y1)
44
42
40
38
36
34
32
30
28
26
24
22
20
-1 0 1 2 3 4 5 6 7 8 9 10 11
Tem p (C)(X1)

10/3/2002 3:00:49 PM
Predicted Gloss(Y1)
Observed Gloss(Y1)
44
42
40
38
36
34
32
30
28
26
24
22
20
70 80 90 100
Tim e (Min)(X2)

Response Plots

10/3/2002 2:55:37 PM
42.3
39.9
37.5
35.1
32.7
30.3
27.9
25.5
23.1
20.7

10/3/2002 2:56:45 PM
42.3
39.9
37.5
35.1
32.7
30.3
27.9
25.5
23.1
20.7


RESIDUAL ANALYSIS
Residual (error) analysis is a useful way of testing/checking whether

many regression analysis assumptions have been violated. There are
several ways to examine residuals.
1. Plot the residuals versus each "independent"

variable.
2. Plot the residuals versus other potential

"independent" variables.
3. Plot the residuals versus the "dependent"

variable.
In the above plots any statistically meaningful non-random patterns in

the residuals indicate problems with the regression analysis.
Another useful way of using residuals to troubleshoot a regression

analysis is to make a probability plot of the residuals. In this instance,
if the normality assumption is reasonable, one would expect to see
something resembling a straight line on the plot.

Let’s examine some of the typical patterns...
1. RANDOM PATTERN
Residual
Predicted y
In this case, residuals are randomly scattered about the center line or
zero residual. The model is likely to be good (provided the other
measures of performance are good as well).
2. FUNNEL
Residual
Predicted y
This pattern occurs when the model has nonconstant variance. The
model, therefore, is inadequate. Transformations of the independent or
dependent variables are typically needed.
3. BENDING PATTERN (UPWARDS OR DOWNWARDS)
Residual
Predicted y
This pattern typically occurs when the errors are not independent.
Again, transformations of the independent or dependent variables may
be needed.
IF THE MODEL IS NOT ADEQUATE...
1. Abandon the model, go back to the problem definition and

develop a new, better model.
2. Transform the data and see if that does the trick
Many times, the model is inadequate because some important variable

has been left out.

EXAMPLE (PLASMA)
Problem: A Simple Transformation
Description: The level of polyamine in plasma of healthy children is

assumed to be a function of age.
X Y
(Age) (Level)
.000 17.000
.000 11.200
1.000 9.200
1.000 12.600
2.000 7.400
2.000 10.500
3.000 8.300
3.000 5.800
4.000 4.600
4.000 6.500
5.000 5.300
5.000 3.800
6.000 3.200
6.000 4.500

A SIMPLE PLOT OF Y VS. X
How good is the linear fit?

Stat-Studio Output
Project: Plasma
Analysis: RS Model 3
Response Model: Plasma Level (Y1)

Model Terms : Intercept, X1


Labels
Bound Bound
Intercept 12.8696 0.9048 10.5534 15.1859
X1 Age -1.6732 0.2509 -2.3156 -1.0308


X1 156.7802 1 156.7802 44.4611 2.3001E-05 Significant
Residuals 42.3148 12 3.5262
Lack of Fit 8.0098 5 1.602 0.3269 0.8817 No LOF
Pure Error 34.305 7 4.9007
Total 199.095 13
Coefficient of Determination R^2: 0.7875


Response Response
Internal External
1 17 12.8696 4.1304 0.2321 2.5101 3.4872 0.9524 4.6966
2 11.2 12.8696 -1.6696 0.2321 -1.0147 -1.016 0.1556 -1.3684
3 9.2 11.1964 -1.9964 0.1429 -1.1483 -1.1653 0.1099 -1.1653
4 12.6 11.1964 1.4036 0.1429 0.8073 0.7948 0.0543 0.7948
5 7.4 9.5232 -2.1232 0.0893 -1.1848 -1.2072 0.0688 -0.9259
6 10.5 9.5232 0.9768 0.0893 0.5451 0.5284 0.0146 0.4053
7 8.3 7.85 0.45 0.0714 0.2487 0.2387 0.0024 0.1622
8 5.8 7.85 -2.05 0.0714 -1.1329 -1.1478 0.0494 -0.7798
9 4.6 6.1768 -1.5768 0.0893 -0.8799 -0.871 0.038 -0.668
10 6.5 6.1768 0.3232 0.0893 0.1804 0.1729 0.0016 0.1326
11 5.3 4.5036 0.7964 0.1429 0.4581 0.4425 0.0175 0.4425
12 3.8 4.5036 -0.7036 0.1429 -0.4047 -0.3901 0.0136 -0.3901
13 3.2 2.8304 0.3696 0.2321 0.2246 0.2155 0.0076 0.2903
14 4.5 2.8304 1.6696 0.2321 1.0147 1.016 0.1556 1.3684
The residuals...
Project Name : Plasma

Plot Name : Residual Plot for Plasma Level(Y1) [RS Model 3]
3/23/2004 2:14:24 AM
1
-1
-2
-3
2 3 4 5 6 7 8 9 10 11 12 13
Predicted Response
Doesn't look good for the home team...

WARNINGS....
Transforming the dependent variable's scale can have a pervasive

impact on a regression analysis. Depending on the transformation
selected, two consequences may arise:
1. The back-transformed regression model will not be

equivalent to the regression model developed from the
untransformed data.
2. Most of the statistics from the transformed regression model

cannot be directly compared (bigger, smaller, same) to their
counterparts from the untransformed regression model. In
the situation where (1) takes place, even the back-
transformed statistics (R-square, root mean square error,
etc) will not be equivalent to those from the regression model
developed from the untransformed data.

If we transform the dependent variable Y
Y ′ = ln(Y )
and run the regression model once again...
The new residuals....
Project Name : Plasma

Plot Name : Residual Plot for Plasma Level(Y1) [RS Model 2]
3/23/2004 2:16:38 AM
1
-1
-2
-3
1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7
Predicted Response
Is it better?

MULTICOLLINEARITY
−1
We have seen the importance of the matrix ( X ′X ) . Some problems
that may arise when computing the matrix are:
1) If the variables are substantially different in magnitude, the

−1
matrix ( X ′X ) may be ill-conditioned. The remedy is to
transform the variables and reformulate the model.
−1
2) If any two variables are highly correlated, the matrix ( X ′X )
will have a determinant which is close to zero.
• This leads to magnification of the variance and the
coefficients.
• Large variances lead to wider confidence intervals
• Unstable estimates
• Erroneous conclusions regarding significance of the
coefficients

DETECTING MULTICOLLINEARITY:
⇒ TYPICAL: High R2, but few significant coefficients

⇒ High pairwise correlations among explanatory
(independent) variables
⇒ High partial correlations
Multicollinearity is also indicated by
1. Large Variance Inflation Factors (VIF)
2. Very small (or zero) eigenvalues of the matrix
X ′X
3. Large condition number

VARIANCE INFLATION FACTORS
Consider the regression model
y = f ( x1 , x 2, ..., x p )
1. Regress x1 on the remaining x’s and obtain the coefficient of

2
determination, say, R1 . Repeat for all the other variables.
2. Compute the variance inflation factor, VIF, for each variable:
1
VIFi =
1 − Ri2
A rule-of-thumb (there’s no agreement among
statisticians!) is that if VIF exceeds a value of 10, there’s
strong reason to suspect multicollinearity.
In the Gloss Properties example, no there is no evidence of

multicollinearity:
Variance
Variable Inflation
INTERCEP 0.00000000
X1 1.12684384
X2 1.12684384

EIGENVALUES AND CONDITION NUMBERS

(FOR THE MATRIX X’X)
These indicators are typically available in good software programs (for

example, SAS).
1. Eigenvalues close to zero indicate multicollinearity.
2. Condition number:
Largest Eigenvalue
CN i =
Eigenvalue i
Typically, if CNi > 30, multicollinearity exists (again, there’s no

consensus among statisticians about this).
For the example,
Collinearity Diagnostics(intercept adjusted)
Variance Variance
Condition Proportion Proportion
Number Eigenvalue Number X1 X2
1 1.33551 1.00000 0.3322 0.3322

2 0.66449 1.41768 0.6678 0.6678
Î NO MULTICOLLINEARITY PROBLEMS ARE

APPARENT

IS MULTICOLLINEARITY NECESSARILY BAD?
The answer truly depends on the objective of the model.
If the objective is
to use the model
for:
Forecasting ===== > Multicollinearity may not be

bad, provided assumptions
regarding the process today
hold in the future as well
Parameter ===== > High degree of

Estimation multicollinearity can be a
problem!

POSSIBLE REMEDIAL ACTIONS
1. Drop a variable from the model (very carefully, since

this may introduce bias in the model)
2. Obtain new data
3. Rethink the model — use experimental design

techniques!
4. Use prior information about some parameters
Transform variables

THE CORRELATION COEFFICIENT
The sample correlation coefficient or Pearson Product-Moment

Correlation Coefficient, r, is a statistic which ranges from -1.0 to 1.0
in value.
sxy =
∑(x i − x )( y i − y )
Let
n −1
then
s xy
r =
sx s y

Some Facts About The Sample Correlation Coefficient:
• −1 ≤ r ≤ 1 for any set of data.
• r provides a measure of the strength of a linear relationship

between the two variables in question.
• As |r| gets closer to one, this indicates a relatively stronger

linear trend component in the data.
• If |r| = 1, all the data fall on a line.
• As |r| gets closer to zero, this indicates a relatively weaker

linear trend component in the data.
• The sign of r is the same as the sign of the slope of the linear
trend component.
Regardless of the strength of the measured linear

relationship between two variables, this does not
establish whether there is a cause-effect
relationship between the variables.

Examples: Relationship of The Correlation Coefficient / Data

r = 1 r = -1
Y | Y |
45 + 100 + A
| A | AA
| AA | AA
40 + A | A
| AA | AA
| AA 80 + A
35 + A | AA
| AA | AA
| AA | A
30 + A | AA
| AA 60 + A
| AA | AA
25 + A | AA
| AA |
| |
20 + 40 +
--+---+---+---+---+---+-- --+---+---+---+---+---+-
10 12 14 16 18 20 10 12 14 16 18 20
X X
r cannot be calculated r = 0.98
Y | 2000 + A
| |
| | A
| | A
| Y | A
| | A
| | A
| | A
15 + AAAAAAAAAAAAAAAAAAAAA 1000 + A
| | AA
| | A
| | AA
| | AA
| | AA
| |AAAA
| |
| 0 +
--+---+---+---+---+---+-- -+---+---+---+---+---+-
10 12 14 16 18 20 10 12 14 16 18 20
X X

MODEL BUILDING

MOTIVATION
We assume there is a relationship between the response and the

independent variables of the type
Y = φ( x1 , x2 ,..., x n ) + ε
The function φ is continuous in the region of interest. In building an
empirical model, the typical steps are:
START
Experimental RAW
Design DATA DATA
Strategy
data
structure
MODELS
y = b0 + x ′b + x ′Bx
subset
model
regression
ANALYSIS
selection
Best
Model

The strategy is to develop models which provide an approximation of

the function f using simple polynomials, e.g.,
k k k −1 k
y = b0 + ∑ bi xi + ∑ b x + ∑ ∑ bij xi x j 2
ii i
1 1 i =1 j >i
Of course, the typical assumptions of convexity and continuity apply.
It is important to note that regression equations can be based on

theoretical or empirical models. For example, we may have a kinetics
equation of the type
⎛ E ⎞
−⎜ ⎟ m
W = K 0e ⎝ RT ⎠
[
∏ iA ] p i
which becomes
m
E
ln W = ln K 0 − + ∑ pi ln A i
RT 1
Depending upon the assumptions we make regarding E and RT, this

can represent a linear model in the coefficients

EXAMPLE (POLYMER)
POLYMER EXAMPLE
Suppose we are trying to maximize the density (g/ml) of a polymer
which is a function of time and temperature.
The operating ranges are:
Time: 2 - 60 minutes
Temperature: 155 - 240 °C
A Central Composite Design is a good choice. This example

will give us an opportunity to describe SAS as a programming
and analysis tool.

THE DATA
Obs X1 X2 Y
1 60 190 101 Star point

2 50 155 72 Corner point
4 30 140 70 Star point
5 30 240 Star point: missing!
9 30 170 91 "Extra" point

10 30 190 98 Center point
• We are missing an observation along the star axis, but we have an

"extra" point that perhaps we could use as well.
• Notice that the three replicates of the center point are placed last in
the data set to allow SAS to compute lack-of-fit statistics.
The highlights of the program will be described.

INPUTTING THE DATA IN SAS
Comments
*********************************************;
* Original File :CCD1.SAS ;
* Study Type : Response Surface ;
* Factor Coding : Coded ;
*********************************************;
DATA Exp2;
INPUT
DSN_ID /* Design ID Points (optional)*/
X1 /* Reaction Time (min) */
X2 /* Reaction Temp (Celsius) */
Y /* Yield (g/ml) */
;
CARDS;
1 60 190 101
2 50 155 72 Variable names
3 50 225 101
4 30 140 70
5 30 170 91
6 10 155 70
7 10 225 83 Data
8 2 190 70
9 30 240 .
10 30 190 98
11 30 190 103
12 30 190 97
;
RUN;
title 'CCD Example with a Missing Point ';
title2 'Input Data';
proc print data=exp2;
run;
Obtain echo
report

FINDING THE ‘BEST’ MODEL...
WILL BE COVERED IN RESPONSE SURFACE
USEFUL TRANSFORMATIONS
In some situations, simple transformations may help improve the

model. Usually, these transformations are applied to the dependent
variable Y, the independent variable X, or both.
Y′ = Y
Y ′ = log10 (Y )
Y ′ = ln( Y )
1
Y′ =
Y
The same transformations can be applied to X.

As indicated previously, if the residuals show a funnel pattern, a log

transformation can often fix the problem. If the error model shows
fanning in the magnitude of the errors about the model in either
direction, then a transformation or a weighted regression analysis is in
order. If the fan widens from left to right, then a logarithmic or a
power transformation (<1) may be useful. If the fan narrows from left
to right then an exponential or power transformation (>1) may be in
order.
Here, Y'=ln(Y) would be indicated.

TRANSFORMATIONS OF THE INDEPENDENT

VARIABLES
Some examples of transformations of the independent variables are

(monotonic transformations):
X1 = X + 100 (addition of a constant)
X2 = X / 1000 (division by a constant)
X3 = (X - 52) / 1000 (subtraction and division by constants)
X4 = X3 (power)
X5 = x (a common power)
X6 = log(X) (logarithmic)
X7 = exp(X) (exponential)
It is somewhat less common to use non-monotonic

transformations of the independent variables.

TREATMENT OF CATEGORICAL VARIABLES
When both categorical and continuous variables are present,

regression analysis transforms the categorical variables into
numerical values.
Choice of proper coding (transformation) is very critical for

correctly evaluating the relative importance of these variables
against the continuous variables.
Two level variable represents 1 Degree of Freedom (DOF)
Each additional level of categorical variable represents an

additional DOF and requires an additional pseudo-variable
(PV) in regression analysis. (PV’s are created internally in the
program)
# of DO # of Pseudo Variables for

Levels F regression
2 1 1
3 2 2
4 3 3
5 4 4
Coding for categorical variables
2 Level Variable:
Levels X[1]
A -1
B 1
3 Level Variable:
Levels X[1] X[2]

A 1 0
B 0 1
C -1 -1
4 Level Variable:
Levels X[1] X[2] X[3]

A 1 0 0
B 0 1 0
C 0 0 1
D -1 -1 -1
5 Level Variable:
Levels X[1] X[2] X[3] X[4]

A 1 0 0 0
B 0 1 0 0
C 0 0 1 0
D 0 0 0 1
E -1 -1 -1 -1
Statistical significance of categorical variable is measured as

the cumulative importance of all of its pseudo-variables.
If any one of the levels of a categorical variable has a

significant influence on the response, the categorical variable
is considered significant.

SUBSET SELECTION METHOD
In subset modeling we use the F test to determined statistical

significance.
Let
SSE( x1 , x2 ) − SSE( x1 , x2 , x3 )
(# of Extra Terms)
F=
SSE( x1 , x2 , x3 )
n − p −1
Hypotheses:
H0: Extra terms = 0 (i.e., β3=0)
Ha: At least one new coefficient not equal to zero (i.e., β3≠0)
Decision rule:
Reject H0 if
F > Fα,#newcoeff ,n− p−1 ,
and conclude that at least one β i ≠ 0

EXAMPLE (THERMAL CONDUCTIVITY)
Problem: Thermal Conductivity: Dummy Variables
Description: The copper plates are supplied by two different

manufacturers. Is there a difference between them?
Data: Data are shown below. A qualitative variable X3 (also known as

a dummy variable) has been added.
The model variables are:
y = Thermal Conductivity (scaled variable with values

from 0 to 100)
X1 = % Copper
X2 = Temperature
X3 = -1 if the plate came from manufacturer A
= 1 if the plate came from manufacturer B

X1 X2 X3 Y
(% Copper) (Temp) (Manufacturer) (Thermal Conductivity)
7 100 B 43
1 86 A 23.7
5 82 B 34.3
8 86 B 35.8
10 84 B 38
0 75 A 22.2
1 80 A 23.1
6 83 A 30
6 91 B 33
9 88 B 38
2 73 A 26.6
10 75 B 36.2
5 81 A 31.6
6 74 A 29
8 87 B 34
4 79 A 30.1
6 94 B 33.9
3 70 A 28.2
3 89 A 30
Coding for categorical variables
Levels X3[1]
A -1
B 1

Project: Thermal Conductivity Modeling

Results
Response Model: Thermal Conductivity (Y1)

Model Terms : Intercept, X1, X2, X3

Model Coefficient Standard 95% Confidence Interval

Labels
Terms Estimate Error
Lower Bound Upper Bound
Intercept 9.0851 7.9825 -10.6546 28.8247
X1 %Copper 1.1476 0.2976 0.4116 1.8835
X2 Temp 0.1969 0.0899 -0.0254 0.4193
X3 Manufacturer 1.1402 0.9933 -1.3161 3.5965

X1 85.397 1 85.397 14.8695 0.0014 Significant
X2 27.5578 1 27.5578 4.7984 0.0436 Significant
X3 7.5675 1 7.5675 1.3177 0.2679 Insignificant
Residuals 91.8895 16 5.7431
Pure Error 0 0 0
Total 599.7855 19


HOW DO WE INTERPRET THE NEW COEFFICIENTS?

TREATMENT OF CATEGORICAL VARIABLES
If X3 = -1 (i.e., plate came from A), then
E ( y ) = β 0 + β 1 x1 + β 2 x 2 + β 3 ( −1)
If X3 = 1 (i.e., plate came from B), then
E( y ) = β 0 + β1 x1 + β 2 x 2 + β 3 (1)
Once the regression equation is obtained, the estimated coefficient b3

represents the effect of changing manufacturers.

POLYNOMIAL EXPANSIONS
One way in which transformations of the independent variables are

constructed is through the use of polynomial expansions. For
instance:
y = β 0 + β1 x + β 2 x 2 + β 3 x 3 + ε
Polynomial expansions:
• require well-structured or designed data

• are useful tools for development of empirical models of
processes
• will interpolate well over the region investigated by the data
However, polynomial models rarely extrapolate well!.

USEFUL HINTS WHEN PERFORMING

REGRESSION ANALYSIS
Obtaining useful regression models is much more complex than simply

knowing how to request regression from a software package. Some tips:
1. Graph, both the data and the model as part of a routine analysis.
2. Ask yourself if the model even makes sense!
3. Check key assumptions! Make sure they are at least crudely met.
4. Analyze the residuals to detect many assumption related

problems.
5. Use confidence intervals to obtain estimates of the model's

reliability.
6. Determine if the model is reliable enough for its intended purpose.
7. Statistical significance and/or a nice fitting model does not

necessarily imply the existence of a cause-effect relationship.
8. Use transformations as a mechanism to better meet assumptions,

not simply as a mechanism to linearize a general nonlinear model.
9. Be extremely careful in evaluating or comparing R-square(s).

When performing regression analysis (cont.):
10. Beware of unnecessary multicollinearity. Regression analysis can

not compensate for poor data structure. If possible design/collect
data which is well structured.
11. Watch out for both extrapolations and hidden extrapolations.
12. Watch out for of data points which are relatively far away from
the rest of the data.
13. Avoid any software package (for purposes of performing

regression) which fails to provide options to assess the quality of
the estimated regression model.
14. Be acutely aware of the fundamental difference between a

confidence limit for individual observations (CLI) and one for the
underlying process average (CLM).
15. Evaluate a regression model as a whole whenever possible.
16. Never force fit a regression analysis into software. Find software
which fits your regression analysis/problem well.
17. Use experimental design techniques to obtain data collections

which are well structured for subsequent regression analysis.

ANALYZING A TYPICAL
COMPUTER REPORT

I. SAS OUTPUT
Typical Regression Analysis

(Boldfaced Numbered Items Are Noted In Section II, "The Language of Regression")
Dependent Variable: Y (1)
ANALYSIS OF VARIANCE
(15)
(14) Sum of Mean (3) (4,6)
Source DF Squares Square F Value Prob>F
Model 1 141.49255 141.49255 24.201 0.0012

Error 8 46.77177 5.84647
C Total 9 188.26432
Root MSE (8) 2.41795 R-square (11) 0.7516

Dep Mean (9) 17.61776 Adj R-sq (17) 0.7205
C.V. (10) 13.72449
PARAMETER ESTIMATES
(12,13) (16) (5)
(14) Parameter Standard T for H0: (6)
Variable DF Estimate Error Parameter=0 Prob > |T|
INTERCEP 1 9.918431 1.74186115 5.694 0.0005
X (2) 1 0.241358 0.04906165 4.919 0.0012
Dep Var Predict Std Err Lower95% Upper95% (7)

X Y Value Predict Mean Mean Residual
10 11.6876 12.3320 1.319 9.2910 15.3731 -0.6444

17 14.0289 14.0215 1.058 11.5821 16.4609 0.0074
20 16.1190 14.7456 0.962 12.5271 16.9641 1.3734
21 12.8660 14.9870 0.933 12.8353 17.1386 -2.1209
25 15.9344 15.9524 0.836 14.0241 17.8807 -0.0179
30 18.4600 17.1592 0.770 15.3829 18.9355 1.3008
39 20.5265 19.3314 0.840 17.3938 21.2690 1.1951
45 23.3330 20.7795 0.999 18.4762 23.0829 2.5535
52 17.2046 22.4691 1.248 19.5915 25.3466 -5.2645
60 26.0174 24.3999 1.576 20.7645 28.0353 1.6175
Sum of Residuals 3.552714E-15

Sum of Squared Residuals 46.7718
Predicted Resid SS (Press) 82.5421

II. THE LANGUAGE OF REGRESSION
This briefly defines some of the language/terms typically encountered when

performing a computer generated regression analysis:
regression analysis - A method of estimating the assumed functional

relationship between two or more quantitative variables.
dependent variable (1) - Identifies the measurement which is to be modeled as a

function of the independent variable(s). Sometimes called a response variable.
These naming conventions are often used regardless of whether the "dependent"
variable truly depends on the independent variable(s).
independent variable(s) (2) - Identifies the measurement(s) which are to be

used to explain the observed variability in the dependent variable. This naming
convention is often used regardless of whether the "independent" variables are
statistically independent of one another.
F-statistic (3) - A signal-to-noise ratio. If this ratio is large statistically,

statistical significance is indicated (i.e., strong evidence that at least one of the
coefficients of the independent variables is not zero). F-statistics are typically
provided with an associated p-value (4).
t-statistic (5) - A transformed version of the F-statistic. Used for assessing

significance of individual coefficients of the independent variables.
p-value (6) - Used to judge whether the effect associated with a given F or t
statistic was statistically significant or not. Smaller p-values indicate stronger
statistical evidence of a given result (leading to the conclusion that the associated
coefficient is not zero). The decision criterion is that a result is statistically
significant when p-value < α (usually α= 0.05).
residual (7) - (observed value - predicted value) = error in prediction. Very useful
in detecting problems with regression models.
standard deviation (8) - A statistical measure of the variability in the dependent

variable which is not unaccounted for by the model. An alternative name is root
MSE (mean square error).
mean (9) - the arithmetic mean of the dependent variable, i.e., "y"

coefficient of variation (10) - Typically expressed as a percentage. The ratio of

the root MSE to the mean of the dependent variable.
R-square (11) - The square of the multiple correlation coefficient. Also called the
coefficient of determination. The percentage of the variation (1=100%) in the
dependent variable which can be explained through use of the regression model.
Ranges between 0 and 1.
parameters (12) - The constants (coefficients) in the model which are to be

estimated.
estimate (13) - The predicted value of a parameter.
ordinary least squares - A commonly used criterion for the estimation of

parameters in a regression model. The model is fitted in a fashion which minimizes
the sum of the squared errors in prediction (Figure 1, 6-2).
degrees of freedom (df) (14) - A statistical measure of how much of the

available data was utilized in estimation of the sum of squares associated with the
model, its error, or one of its coefficients.
sum of squares (15) - Associated with a particular partitioning of data variation.

An early step in developing the F or t statistic.
standard error of estimate (16) - The standard deviation with which the
parameter in question was estimated. A measure of uncertainty of the estimated
value.
adjusted R-square (17) - A method which adjusts the traditional definition of R2

to account for degrees of freedom. This adjustment is theoretically flawed, but is
still found useful by some. Its use is not recommended unless one is familiar with
its definition and limitations.

III. DETAILED EXAMPLE: REGRESSION ANALYSIS IN SAS
All examples and figures provided in this section of the seminar were constructed
with the use of SAS/PC the PC based version of SAS (Statistical Analysis System,
SAS Inc., Cary, NC). SAS software runs in many computing environments (PC,
IBM Mainframe, VAX, HP, ...).
The following program illustrates how to read in a simple set of data into SAS, print
the data, request a regression analysis with confidence limits, perform a graphical
residual analysis, and examine a plot of the model.
* SAS program illustrating regression analysis. All statements ;

* in SAS end with a semicolon. Comments start with a *. ;
* =================== DATA INPUT ===============================;

data one; * The data statement is used to create a SAS dataset.;
* This will make a temporary SAS data set named one. ;
input y x; * The input statement describes the data format. ;

* SAS expects two columns to be input here, Y and X. ;
;
cards; * <======= The cards statement indicates the data follows.
11.69 10
14.03 17
16.12 20
12.87 21
15.93 25
18.46 30
20.53 39
23.33 45
17.20 52
26.02 60
; * < ===== A semicolon is used to indicate the end of the data.;
* ==============REQUESTING AN ECHO REPORT =====================;
proc print data=one; * Requests that dataset one be printed. ;
title 'SAS Example'; * Defines title to go above printout. ;
* ================= REGRESSION ANALYSIS =========================;

proc reg data=one; * Requests an ordinary least squares ;
* regression analysis be performed. ;
model y=x / p cli; * Specifies model, y is "dependent" var.;

* x is "independent" var. Also requests;
* p, cli options (predicteds, conf lim).;
output out=one r=res; * Adds variable res (residuals) to dataset one;

* =================== PLOTS =====================================;
proc gplot data=one; * Requests a high resolution plot. ;
plot y*x / vaxis=10 to 30 by 5 haxis=10 to 60 by 10;

* Specifies plot / plot options (axis scalings). ;
symbol v=diamond i=rlclm; * v = plot symbol, i=interpolation ;

* method (linear regression with limits);
proc gplot data=one; * Requests a high resolution plot. ;
plot res*x / vref=0; * Plot residuals vs x, draws ref. line. ;

symbol v=square; * Use square as the plot symbol. ;
run; * Tell SAS to run the previous lines. ;

IV. MORE EXAMPLES: LINEAR REGRESSION

ANALYSIS IN SAS
This example illustrates SAS code which does the following tasks:
1. Reads in the data

2. prints the data
3. specifies a regression analysis to fit Yield = a + b*Flow, and
4. provides predicted values and confidence limits on the individual
observations.
data; * Reads the data into SAS;

input yield flow hour;
cards;
55 10.5 1
58 11.9 2
64 12.6 3
69 13.2 4
73 13.9 5
proc print; * Prints all variables in data set;
title1 'Example of SAS Code'; * Specifies first title;
proc glm; * Specifies regression model to be fit with / options;
model yield=flow / p cli;
title 'Intercept in Model, CLI';
If confidence limits for the mean were desired one would simply change the cli in
the above to clm instead. If a no-intercept regression was desired as well the option
list would have been: / p clm noint
If a multiple regression model was desired one would simply add the additional
independent variable names in the model statement.
For example: MODEL YIELD = FLOW HOUR / P CLM;
If one wanted to perform a stepwise regression analysis, the procedure stepwise

would be used. The SAS code for stepwise regression using the maximum R-square
improvement method would be:
PROC STEPWISE;
MODEL YIELD = TEMP RPM FLOW FEED / MAXR;

SECTION III
Factorial and Screening

Designs
Experimental Design Des-1
Table of Contents
1. INTRODUCTION..............................................................................................................................................2
2. A PHOTOGRAPHIC PROCESS ..............................................................................................................4

PROBLEM DISCUSSION ...........................................................................................................................................5
3. FACTORIAL DESIGNS ............................................................................................................................10
ILLUSTRATION OF EFFECTS AND INTERACTIONS..............................................................................................11
EXAMPLE OF A 22 DESIGN ...................................................................................................................................12
FULL-FACTORIAL DESIGN ...................................................................................................................................13
EXAMPLE (YIELD) ...........................................................................................................................................18
Coding ..............................................................................................................................................................20
RANDOMIZATION.......................................................................................................................................27
ADVANTAGES OF FULL FACTORIAL DESIGNS..............................................................................29
4. SCREENING DESIGNS.................................................................................................................................31
PLACKETT-BURMAN DESIGNS ............................................................................................................................33
Generation of Plackett Burman Design.....................................................................................................34
5. COMMON DESIGN/ANALYSIS PITFALLS...........................................................................................42
_____________________________
1. Introduction
Experiment:
A study to assess the effect of one or more factors

on some response.
Essential for an experiment:
• Knowledge of the process

• A response variable
• Clear GOALS and OBJECTIVES
Why Statistical Experimental Design:
• A structured plan of attack

• More efficient
• Meshes with powerful analysis tools
• Forces experimenter to organize
_____________________________
Objectives of R&D in the Chemicals Industry
Mostly
• improving existing products/processes
• developing new products/processes
• developing new applications (“fishing”)
Frequently
• fundamental models: “scale-up” or “optimizing”
Mostly interested in understanding the

trends in the influence of various control
variables on the properties of interest.
control
control Industrial R&D resulting
resulting
variables
variables response
response
[x1,..,x
XX==[x ,..,xn]] Semi-Black Box yy
1 n
Approach
y = Xβ + ε
_____________________________
2. A Photographic Process
The concept phase
Exposed film is developed using specially formulated

solution of chemicals. The idea is to convert silver
halide crystals that have been exposed to light into
black metallic silver. The real trick is to make the
silver image black in proportion to the amount of light
striking on the film. It is further possible to roll up the
elements of image quality into a single value metric
called the IQ (Image Quality) of the picture.
_____________________________
Problem Discussion
• How many factors can be studied ?
• resources, constraints in Lab and Plants
• Which factors are to be studied ?
• What factors are most important ?
• What model to be used ?
• linear, polynomial, theoretical (first
principles)
• Which experimental design strategy?
• One-at-a-time, Taguchi, Factorial, Box-
Behnken, ...
_____________________________
Goal:
To obtain a maximum IQ near 100 with the
least variation
Objective:
To find an empirical model of IQ as a
function of
» Temperature
» Ph
» Humidity
» Speed
» Metol conc
» HQ conc
» KI conc
Optimize the response surface
_____________________________
Regression Analysis
Variance of coefficients:
^
s ( β ) = MSE ( X ′X ) −1
2
Depends upon the Depends upon the

experimental set-up, experimental design
procedure, skills, used for modeling.
instrumentation.
Minimize for best modeling results
Statistical experimental designs maximize

the model precision through the design
matrix
_____________________________
The
The Design
Design Pyramid
Pyramid
Hierarchy of Experimental Design and Models:

(not all inclusive - variance components)
Level
Example
Design Model
theoretical
empirical
(response surface)
screening
_____________________________
The
The Design
Design Pyramid
Pyramid
The four facets of the Pyramid
Level Model Design Examples

Screening Linear Fractional Taguchi
Factorial Plackett-Burman
Empirical Polynomial Response Box-Behnken

Surface Central Composite
Theoretical Non-linear Stagewise D-optimal
_____________________________
3. FACTORIAL DESIGNS
Factorial experiments are a series of experiments in
which several factors are controlled in order to
investigate their effects on some dependent variable.
Definitions
1. Independent Variable (Factors: x1, x2, ..., xn) -

Any feature of the experimental conditions which
may be assigned at will from one trial to another.
2. Levels [ (-1, +1) or ( -, + ) ]- The various values of

the independent variables which are used for an
experimental run.
3. Dependent Variable (Response; y) - The

numerical result of an experimental run.
4. Effect - The change in the dependent variable

produced by a change in the level of an independent
variable.
5. Interaction - If the difference in the dependent

variable between two levels of an independent
variable is not the same for each level of another
independent variable, there is an interaction
between the two independent variables.
_____________________________
Illustration of Effects and Interactions
y
X2 = +1
X2 = -1
X1
Figure 1. No Interaction
y
X2 = +1
X2=-1
X1
Figure 2. Interaction
_____________________________
Example of a 22 design
Two independent variables are given below.
X1
-1 +1
-1 103 125
X2
+1 198 142
The effect of each independent variable alone, called the

main effect, and the interaction effect are calculated in
the following manner:
Effect Calculation
x1 1/2[(125-103) + (142 - 198)] = -17
x2 1/2[(198-103) + (142 - 125)] = 56
x1x2 or x2x1 1/2 [(142 - 198) - (125 - 103)] = -39

1/2 [(142 - 125) - (198 - 103)] = -39
The multiplier of 1/2 is for averaging the two measures

of the effect.
_____________________________
Full-Factorial Design
A full factorial design is one in which
all possible combinations of the factors at
all levels involved in the experiment are
used. The number N of all possible
combinations to be performed during the
experiment is given by
N = nk
Where
n: the number of levels, and
k: the number of factors.
_____________________________
A three-factor design...
Run A B C
1 +1 +1 +1
2 +1 -1 -1
3 -1 +1 -1
4 -1 -1 +1
• A design is considered to be balanced vertically when

the transformed values, Xj, for each factor column
sum up to zero for all factors j
N
∑ x = 0
ji
i=1
• A 2-level design matrix is said to be orthogonal if it is
balanced and if the sum of every dot product for all
possible variable pairs is zero,
N
∑ x ji xui = 0 for all j ≠ u
i =1
_____________________________
• The reason for orthogonality is that it

allows the desired effects to be
evaluated neatly. A high degree of non-
orthogonality results in a dependency of
effects which is particularly
undesirable.
• For example, if effects A and B are

dependent and only A is important, B
may appear important due to the
dependency with A. Even worse, if A
and B are both important, but one in a
positive direction and the other in a
negative direction, their dependency
may cause both to appear unimportant.
_____________________________
FULL FACTORIAL DESIGNS
Suppose we wish to examine 3 factors (A, B, and

C) each at two levels. The set of all possible
effects for this example is
2-Way 3-Way
Main Effects Interactions Interactions
A AB ABC
B AC
C BC
• The number of runs (N) for a full factorial

design of k factors is N = 2 . k
• The number of orthogonal columns representing

all possible effects is N - 1.
A total of 7 effects could be analyzed in 8

runs!
(However, there is no such

thing as a free lunch..)
_____________________________
3 FACTORS AT TWO LEVELS
Run A B C AB AC BC ABC
1 -1 -1 -1
2 -1 -1 +1
3 -1 +1 -1
4 -1 +1 +1
5 +1 -1 -1
6 +1 -1 +1
7 +1 +1 -1
8 +1 +1 +1
What would be the resulting

equation?
_____________________________
EXAMPLE (YIELD)
Problem: A researcher is investigating a reaction in

which he wants to study the effects of certain operating
conditions on the yield of the reaction. The operating
conditions which will be included are pressure,
temperature, and reaction time.
This is a 23 design...
Choice of levels to use for the independent variables:
1. The levels should be far enough apart to cause a

significant change in the dependent variable.
2. The levels should be set so that they can be

controlled to ensure that a given level of a factor is
reasonably constant throughout an experimental run
for each experimental run that it appears at the
level.
3 The levels represent the operating region of interest.
4. Each combination of factors (treatment) can be

achieved physically.
_____________________________
For the example problem, the following levels of the

independent variables were selected:
Level
Independent
Variable Low High
z1: Pressure (KPa) 2 6
z2: Temperature (ºC) 100 200
z3: Time (min.) 10 20
_____________________________
Coding
Notice that the pressure and temperature are measured

in different units here. The coded regression equation
will be determined by
⎜⎛ zi −
⎛ High + Low ⎞ ⎟⎞
⎝ ⎝ 2 ⎠⎠
xi =
⎛ High − Low ⎞
⎝ 2 ⎠
where zi is the uncoded independent variable ('raw

data').
_____________________________
For example, we have
⎛ ⎞
⎜ ⎟
⎜ z1 − 4 ⎟ = ⎛⎜ z1 − 4 ⎞⎟
Pressure: ⎜ ⎛ 6 − 4⎞ ⎟ ⎝ 2 ⎠
⎜ ⎜⎝ ⎟⎟
⎝ 2 ⎠⎠
⎛ ⎞
⎜ ⎟
⎜ z 2 − 150 ⎟ = ⎛⎜ z 2 − 150 ⎞⎟
Temperature: ⎜ ⎛ 200 − 100 ⎞ ⎟ ⎝ 50 ⎠
⎜ ⎜⎝ ⎟⎟
⎠⎠
⎝ 2
⎛ ⎞
⎜ ⎟
⎜ 3 z − 40 ⎟ = ⎛⎜ z3 − 15 ⎞⎟
Reaction Time: ⎜ ⎛ 20 − 10 ⎞ ⎟ ⎝ 5 ⎠
⎜ ⎜⎝ ⎟⎟
⎝ 2 ⎠⎠
0 0 0
The point with coordinates ( z1 , z2 ,. . . , zk )
zmax + zj
min
j
z =
0
j ; j = 1, 2, .. . , k
2
is called the center point of the design.
_____________________________
Full Factorial Design
Factors on
Natural Coded
Scale Factors Yield
Obs. z1 z2 z3 x1 x2 x3 y
1 100 2 10 -1 -1 -1 2
2 200 2 10 1 -1 -1 6
3 100 6 10 -1 1 -1 4
4 200 6 10 1 1 -1 8
5 100 2 20 -1 -1 1 10
6 200 2 20 1 -1 1 18
7 100 6 20 -1 1 1 8
8 200 6 20 1 1 1 12
Center Points
9 150 4 15 0 0 0 8
10 150 4 15 0 0 0 9
11 150 4 15 0 0 0 8.8
_____________________________
3/17/2005 10:17:27 AM
Results
Project : New Design Project
Starting Response Model : Yield (Y1)
Number of terms in the User Starting Model : 8

Model Terms : Intercept, X1, X2, X3, X1X2, X1X3, X2X3, X1X2X3
Weights : Unity


Labels
Bound Bound
Intercept -9.3591 1.0116 -11.9596 -6.7586
X1 Pressure 0.6705 0.227 0.087 1.2539
X3 Time 0.5023 0.0748 0.31 0.6945
X1X2 Pressure*Temp 0.0078 0.0017 0.0035 0.0121
X2X3 Temp*Time 0.005 0.0004 0.004 0.0061
X1X2X3 Pressure*Temp*Time -0.0009 9.0795E-05 -0.0012 -0.0007

X1 1.8083 1 1.8083 8.7243 0.0317 Significant
X3 9.3476 1 9.3476 45.0979 0.0011 Significant
X1X2 4.556 1 4.556 21.9806 0.0054 Significant
X2X3 32.0026 1 32.0026 154.3985 5.9844E-05 Significant
X1X2X3 21.8312 1 21.8312 105.3258 0.0002 Significant
Residuals 1.0364 5 0.2073
Lack of Fit 0.4764 3 0.1588 0.5671 0.6884 No LOF
Pure Error 0.56 2 0.28
Replication Error 0 0 0
Repeats Error 0.56 2 0.28
Total 174.5818 10
_____________________________

Observed Predicted Weight Standardized Residuals

Response Response s
Internal External
1 2 1.7545 1 0.2455 0.6023 0.8549 0.8275 0.1844 0.9296
2 6 6.5045 1 -0.5045 0.4148 -1.4487 -1.701 0.2479 -1.3072
3 4 3.8455 1 0.1545 0.6932 0.6128 0.57 0.1414 0.7821
4 8 8.0045 1 -0.0045 0.9148 -0.0342 -0.0306 0.0021 -0.0915
5 10 9.9591 1 0.0409 0.7557 0.1818 0.1631 0.017 0.2619
6 18 17.8909 1 0.1091 0.875 0.6777 0.6361 0.5359 1.5364
7 8 8.3227 1 -0.3227 0.5739 -1.0859 -1.1111 0.2647 -1.177
8 12 11.9364 1 0.0636 0.8977 0.4371 0.3986 0.2795 1.0781
9 8 8.5273 1 -0.5273 0.0909 -1.2147 -1.294 0.0246 -0.3735
10 9 8.5273 1 0.4727 0.0909 1.089 1.1153 0.0198 0.3219
11 8.8 8.5273 1 0.2727 0.0909 0.6283 0.5855 0.0066 0.169
_____________________________
DESIGN-EASE Analysis
Yield
18.00
A
c 15.33
t
u
a 12.67
l B-
Y 10.00 B+
i
e
l
d 7.33
B+
B-
4.67
2.00
A- A+
Interaction of A:Pressure and B:Temp
Yield
18.00
A
c 15.33 C+
t
u
a 12.67
l
Y 10.00
i
e C+
l
d 7.33 C-
4.67
C-
2.00
A- A+
Interaction of A:Pressure and C:Time
_____________________________
Yield
18.00
A
c 15.33
t C+
u
a 12.67
l
Y 10.00 C+
i
e
l
d 7.33
C-
4.67
C-
2.00
B- B+
Interaction of B:Temp and C:Time
Actual Yield
8.00 12.00
B+ 4.00 8.00
T
e
m 10.00 18.00 C+
p
e
m
i
T
B- 2.00 6.00 C-
A- A+
Pressure
_____________________________
RANDOMIZATION
In practice, the actual experiment sequence should

be randomized. The setup for the experiment is given
below. Notice we are not including the term x0.
Coded Actual
Experimen Random
t x1 x2 x3 z1 z2 z3 Order
#
1 - - - 2 100 10 2
2 + - - 6 100 10 4
3 - + - 2 200 10 6
4 + + - 6 200 10 1
5 - - + 2 100 20 7
6 + - + 6 100 20 5
7 - + + 2 200 20 8
8 + + + 6 200 20 3
The analysis would proceed exactly as outlined

previously.
_____________________________
Wrapping up factorial designs...
• For 3 factors, each with 3 levels, a full factorial design

consists of 33 = 27 runs.
• Things can get out of hand in a hurry...
Number of levels
Number of Factors 2 3 4
2 4 9 16
3 8 27 64
4 16 81 256
5 32 243 1024
6 64 729 4096
7 128 2187 16384
_____________________________
ADVANTAGES OF FULL
FACTORIAL DESIGNS
1. Orthogonality
2. No aliasing
3. All main factors and all interactions can be

evaluated.
DISADVANTAGES?
cost, time, and resources!
_____________________________
FACTORIAL DESIGN
PROCEDURE
1. Determine independent variables

which might be significant.
2. Select the levels for each

independent variable.
3. Select the appropriate factorial

design.
4. Randomize the sequence of

experimental runs.
5. Conduct the experimental runs.
6. Compute effects.
7. Test effects for significance.
_____________________________
4. SCREENING DESIGNS
• Used at the beginning of the program when
process knowledge is relatively limited
• Simple efficient filters to investigate a large

number of control variables in a few experiments
— 16 experiments to investigate up to 15
variables
• Identify the variables with the greatest effect on

the response
• Fractional factorial designs

— Plackett-Burman or Taguchi
• New and existing Process or Product

Development
_____________________________
What Do Screening Designs Attempt To Do ?
To model the influence of the control variables on

the response variable by a linear model.
response = k + a[variable 1] + b[variable 2] +

c[variable 3] + .... + [error]
The linear model will tell you the minimum

number of experiments you need to run.
minimum number = number of parameters

of experiments in the Linear Model
_____________________________
Plackett-Burman Designs
• Developed by R. L. Plackett and J. P. Burman
during World War II.
• These are RIII designs for screening a large

number of factors.
• Fractional factorial:
2k-p = 4, 8, 16, 32, 64, 128, ... .
• Plackett-Burman Designs:
4, 8, 12, 16, 20, 24, 28, ...
• The Plackett Burman design is based on

Hadamard Matrices
• Main effects are confounded in some way with

the interaction effects if they are present.
• P-B design can nominally screen as many as n-1

factors based on an n experiment design.
Response = b0 + b1(Factor 1) + b2(Factor 2) + ...

+ bi(Factor i)
_____________________________
Generation of Plackett Burman Design
n=8 (+ + + - + - -)
n = 12 (+ + - + + + - - - + -)
n = 16 (+ + + + - + - + + - - + - - -)
n = 20 (+ + - - + + + + - + - + - - - - + + -)
n = 24 (+ + + + + - + - + + - - + + - - + - + - - - -)
Eight-Experiment Plackett-Burman Design
RUN A B C D E F G
1 + + + - + - -
2 + + - + - - +
3 + - + - - + +
4 - + - - + + +
5 + - - + + + -
6 - - + + + - +
7 - + + + - + -
8 - - - - - - -
_____________________________
Twelve-Experiment Plackett-Burman Design
EXP X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11
1 + + - + + + - - - + -
2 + - + + + - - - + - +
3 - + + + - - - + - + +
4 + + + - - - + - + + -
5 + + - - - + - + + - +
6 + - - - + - + + - + +
7 - - - + - + + - + + +
8 - - + - + + - + + + -
9 - + - + + - + + + - -
10 + - + + - + + + - - -
11 - + + - + + + - - - +
12 - - - - - - - - - - -
_____________________________
Advantage of P-B Design
Limited number of runs to evaluate large

number of factors
Disadvantage of P-B Design
The assumption that interactions (if they

exist) are not strong enough to mask main
effects
Note: For most processes and most type

of factors evaluated over reasonable
ranges, these assumption is often fairly
accurate.
_____________________________
Confounding Table For 8 Experiment Plackett-Burman Design

Terms Are Fully Confounded With Each Another
EFFECT CONFOUNDED WITH

A BF CD EG |
B AF CG DE |
C AD BG EF |
D AC BE FG |--> Main effects confounded with
E AG BD CF | interactions
F AB CE DG |
G AE BC DF |
AB CE DG F |
AC BE FG D |
AD BF DG C |
AE BC DF G |
AF CG DE B |
AG BD CF E |
BC AE DF G |
BD AG CF E |
BE AC FG D |
BF CD EG A |
BG AD BG C |-> Interactions confounded with main effects
CD BF EG A | and interactions
CE AB DG F |
CF AG BD E |
CG AF DE B |
DE AF CG B |
DF DE BC G |
DG AB CE F |
EF AD BG C |
EG BF CD A |
FG AC BE D |
_____________________________
Hand Analysis of a Plackett-Burman Design
EXP X1 X2 X3 X4 X5 RESP
1 + + + - + 90
2 + + - + - 85
3 + - + - - 82
4 - + - - + 50
5 + - - + + 91
6 - - + + + 48
7 - + + + - 60
8 - - - - - 64
X1 Effect
(90 + 85 + 82 + 91)/4 - (50 + 48 + 60 + 64)/4 =
87.00 - 55.50 = 21.50 (Rank = 1)
X2 Effect
(90 + 85 + 50 + 60)/4 - (82 + 91 + 48 + 64)/4 =
71.25 - 71.25 = 0.00 (Rank = 5)
X3 Effect = -2.50 (Rank = 3)

X4 Effect = -0.50 (Rank = 4)
X5 Effect = -3.00 (Rank = 2)
_____________________________
Example: Computer Analysis of Plackett-

Burman Design
Identify which controlling variables (X1, X2, X3,

X4, X5, and/or X6) impact the selectivity in a
chemical process. The control variable ranges are
as follows:
X1: 60 to 100
X2: 2 to 8
X3: 0 to 50
X4: 25 to 75
X5: 20 to 50
X6: 10 to 30
DATA
OBS SEL X1 X2 X3 X4 X5 X6
1 86.3093 100 2 50 25 50 10
2 86.3841 60 2 50 75 50 30
3 83.0953 60 8 0 75 50 10
4 85.8008 60 2 50 75 20 30
5 86.0550 100 8 50 25 20 30
6 77.7010 60 8 50 25 20 10
7 88.5746 100 2 0 25 50 30
8 87.9833 100 8 0 75 20 30
9 81.8652 60 8 0 25 50 30
10 86.8871 100 8 50 75 50 10
11 87.4913 100 2 0 75 20 10
12 78.7226 60 2 0 25 20 10
_____________________________
CORRELATION ANALYSIS
7 'VAR' Variables: SEL X1 X2 X3 X4 X5 X6
Pearson Correlation Coefficients / N = 12
SEL X1 X2 X3 X4 X5 X6
SEL 1.00000 0.71987 -0.23475 0.03402 0.44585 0.22667 0.39845
X1 0.71987 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000
X2 -0.23475 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000
X3 0.03402 0.00000 0.00000 1.00000 0.00000 0.00000 0.00000
X4 0.44585 0.00000 0.00000 0.00000 1.00000 0.00000 0.00000
X5 0.22667 0.00000 0.00000 0.00000 0.00000 1.00000 0.00000
X6 0.39845 0.00000 0.00000 0.00000 0.00000 0.00000 1.00000
_____________________________
MODEL1
Dependent Variable: SEL
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Prob>F
Model 6 139.7907 23.29846 49.385 0.0003

Error 5 2.35885 0.47177
C Total 11 142.149
Root MSE 0.68685 R-square 0.9834

Dep Mean 84.73913 Adj R-sq 0.9635
C.V. 0.81055
Parameter Estimates
Parameter Standard T for H0:

Variable DF Estimate Error Param=0 Prob>|T|
INTER 1 68.4260 1.1599 58.992 0.0001

X1 1 0.12388 0.00991 12.496 0.0001
X2 1 -0.26932 0.06619 -4.075 0.0096
X3 1 0.00468 0.00793 0.591 0.5805
X4 1 0.06138 0.00793 7.739 0.0006
X5 1 0.05201 0.01322 3.935 0.0110
X6 1 0.13714 0.01983 6.916 0.0010
_____________________________
5. COMMON DESIGN/ANALYSIS
PITFALLS
Multicollinearity: The failure to control variables as

independently as possible.
Hidden Extrapolation: The failure to properly

identify the region over which responses can be
interpolated.
Overfitting Data: When model complexity matches or

exceeds the complexity of the data used in modeling.
One-at-a-Time: Exploring only one control dimension

at a time.
Unknown Reproducibility: Not knowing

experimental reproducibility and failing to check it out
early in a program.
Too Simple a Model: Limiting model forms to only

include lines and planes.
_____________________________
Plant Window: Looking at too narrow a design

window to resolve the signal from the noise.
Destructive Summarization: Discarding information

by destructively summarizing the data.
The Rack: Collection of a dense grid of experiments as

a design.
Running the Right Experiment: An iterative

process of building poor designs.
Rationalization: Cleverly thinking of reasons why

good design practices don't apply to your situation.
_____________________________
Response Surface RS-1
SECTION IV
Response Surface Concepts
__________________________________
Table of Contents
1. RESPONSE SURFACE CONCEPTS ...........................................................................................3

TYPICAL STRATEGY:..........................................................................................................................7
2. THREE LEVEL STRATEGIES ................................................................................................13
BOX-BEHNKEN........................................................................................................................................14
CENTRAL COMPOSITE DESIGNS............................................................................................................17
NOTES AND SUGGESTIONS: .........................................................................................................20
HOW MANY CENTER POINTS? ................................................................................................22
CODING A CCD:...................................................................................................................................23
FINDING THE ‘BEST’ MODEL........................................................................................................24
3. EXAMPLE........................................................................................................................................25
__________________________________
1. RESPONSE SURFACE
CONCEPTS
We assume there is a relationship between the response

and the independent variables of the type
Y = φ( x1 , x2 ,..., x n ) + ε
The function φ is continuous in the region of interest. As

before, the modeling sequence is
START
Experimental RAW
Design DATA DATA
Strategy
data
structure
MODELS
y = b0 + x ′b + x ′Bx
subset
model
regression
ANALYSIS
selection
Best
Model
__________________________________
The strategy is to develop models which provide an

approximation of the function φ using simple
polynomials, e.g.,
k k k −1 k
y = b0 + ∑ bi xi + ∑ b x + ∑ ∑ bij xi x j
2
ii i
1 1 i =1 j >i
Of course, the typical assumptions of convexity and

continuity apply.
It is important to note that response surface equations

can be based on theoretical or empirical models. For
example, we may have a kinetics equation of the type
⎛ E ⎞ m
∏[ A ]
−⎜ ⎟ pi
⎝ RT ⎠
W = K 0e i
1
which becomes
m
E
ln W = ln K 0 − + ∑ pi ln A i
RT 1
DOE techniques are also useful in these cases.
__________________________________
WHERE DOES IT FIT?
Level Model Design Examples
Screening Linear Fractional Taguchi

Factorial Plackett-Burman
===> Empirical Polynomial Response Box-Behnken

Surface Central Composite
Theoretical Nonlinear Stagewise D-Optimality
__________________________________
Empirical Designs
(Response Surface Designs)
• Applied after screening design

- with important variables known
- relatively deeper process knowledge
• Advantages
- high precision in prediction
- simplified models for complex phenomena
• Drawbacks
- fundamental knowledge may not be
enhanced
- estimation best at the center of the design
• Optimizing new or existing Processes or Products
__________________________________
TYPICAL STRATEGY:
1. Start with a screening design or perhaps a factorial

design (to obtain a rough approximation of the
gradient)
2. Once we are near the optimum, use quadratic models

and test for lack of fit with replicates
Start here with a

Optimum via
screening design
response surfaces
Response (y)
X2
x1
In the region of interest (i.e., the design region), a

second-order model may provide a good approximation
to the true curvature of the surface Y = φ( x1 , x2 ,..., x n ) + ε
__________________________________
y$ = b0 + x ′b + x ′Bx
where, b0, the intercept, is a scalar.
Then
∂y$
= b + 2Bx = 0
∂x
1
x 0 = − B −1 b
2
The predicted response is
1
y$ 0 = b0 + x 0 b
2
The canonical form is
k
y$ = y$ 0 + ∑ λ i wi2
1
where
{wi}: transformed indep vars.

{λi} : eigenvalues of B
__________________________________
Eigenvalue Sign Interpretation
(1) All positive Minimum
(2) All negative Maximum
(3) Some negative, some Saddle point

positive
4) All one sign with one Rising or falling ridge

eigenvalue or more
close to zero
__________________________________
Response
Response Surface
SurfaceShapes:
Shapes: Plane
Plane
3-D
CONTOUR
2.0
61
67
70
74 1.5
RESPONSE
X2
66 1.0
58
64
55
58 0.5
9
52
8
50 0.0
2.00 7 X1 6.00 7.50
1.33
6.75 8.25 9.00
0.67
X2 0.00 6 X1
Response
Response Surface
SurfaceShapes:
Shapes: Twisted
TwistedPlane
Plane
3-D
CONTOUR
2.0 33
38
65 1.5
43
X2
RESPONSE
54 1.0
48
43 0.5 53
9
58
8 63
0.0
32 7 X1 6.00 6.75 7.50 8.25 9.00
2.00
1.33 X1
X2 0.67 6
0.00
__________________________________
Response
ResponseSurface
Surface Shapes:
Shapes: Maxima
Maxima
3-D CONTOUR
55.0
66
75 47.5
X2
70
RESPONSE
65 40.0
66
55
5 32.5
4 62
45 54
58
55 3 X1 25.0 50 58
45
35 2.00 2.75 3.50 4.25 5.00
X2 25 2
X1
Response
ResponseSurface
SurfaceShapes:
Shapes: Stationary
StationaryRidge
Ridge
3-D CONTOUR
70
95.00 55
X2
RESPONSE
85.67 40
70
74
78
82
86
90
90
86
82
78
74
70
94
94
76.33
11 25
8
67.00
70 5 X1
50 10
30 10 2 2.00 4.25
X2 6.50 8.75 11.00
X1
__________________________________
Response
ResponseSurface
SurfaceShapes:
Shapes: Rising
RisingRidge
Ridge
3-D
CONTOUR
65.0
74
76.00 57.5
X2
RESPONSE
66.67 50.0
70
54
58
57.33 66
18.5 42.5
15.5
58
50
62
48.00 54
12.5 X1 35.0
50
65 55 9.50 14.00 16.25 18.50

45 9.5 11.75
X2 35 X1
Response
ResponseSurface
Surface Shapes:
Shapes: Saddle
Saddle
3-D CONTOUR
65.0
60
170 150
75
57.5
RESPONSE
90
130
105
X2
50.0 135
120
90
12
12
0
11 42.5
10
5
50 10 X1
90
65
55
75
45 9 35.0
X2 35 9.00 9.75 10.50 11.25 12.00
X1
__________________________________
2. THREE LEVEL STRATEGIES
• Box-Behnken
• Central Composite
• Useful for process optimization
• Can be expanded to accomodate other control

variables
• Useful if underlying model can be approximated by a

linear or quadratic model
• Both approaches are best suited for quantitative

factors.
__________________________________
Box-Behnken
• Very useful 3-level designs for modeling quantitative

factors
• Near orthogonal...
• Rotatable or nearly rotatable
__________________________________
Three-variable BB design
Run A B C
#
1 + + 0
2 + - 0
3 - + 0
4 - - 0
5 + 0 +
6 + 0 -
7 - 0 +
8 - 0 -
9 0 + +
10 0 + -
11 0 - +
12 0 - -
13 0 0 0
14 0 0 0
15 0 0 0
__________________________________
Four-variable BB design
Run A B C D
#
1 - - 0 0
2 - + 0 0
3 + - 0 0
4 + + 0 0
5 0 0 - -
6 0 0 - +
7 0 0 + -
8 0 0 + +
9 0 0 0 0
10 - 0 0 -
11 - 0 0 +
12 + 0 0 -
13 + 0 0 +
14 0 - - 0
15 0 - + 0
16 0 + - 0
17 0 + + 0
18 0 0 0 0
19 - 0 - 0
20 - 0 + 0
21 + 0 - 0
22 + 0 + 0
23 0 - 0 -
24 0 - 0 +
25 0 + 0 -
26 0 + 0 +
27 0 0 0 0
__________________________________
Central Composite Designs
• Flexible and efficient
• May contain a factorial design as its 'kernel'

(Advantage: start with a factorial and later expand it
to include the star points!)
X2
X3
X1
__________________________________
Two-Variable Central Composite Design
Run A B
#
1 + +
2 + -
3 - +
4 - -
5 0 0
6 0 0
7 0 0
8 α 0
9 -α 0
10 0 α
11 0 -α
__________________________________
Three-variable CCD
Run A B C
#
1 - - -
2 - - +
3 - + -
4 - + +
5 + - -
6 + - +
7 + + -
8 + + +
9 0 0 0
10 0 0 0
11 0 0 0
12 0 0 0
13 0 0 0
14 0 0 0
15 α 0 0
16 -α 0 0
17 0 α 0
18 0 -α 0
19 0 0 α
20 0 0 -α
__________________________________
NOTES AND SUGGESTIONS:
• CCD's can be fractionated
• Need 2k axial points
• Axial points can be located outside of the 'cube' or on

the face
• CCD's can be run sequentially:
- run factorial + center points first
- add axial points if curvature is present
1) If rotatability is desirable, choose:
( )
1/ 4
α = nf
nf is the number of points in the factorial part of a

CCD.
__________________________________
2) If axial points are infeasible, choose:
α=1
3) In general, for orthogonality:
[ ]
1
⎛ F + T − F F ⎞⎟
2 4
α =⎜
⎜ 4 ⎟
⎝ ⎠
where
F=number of factorial points
T=number of axial + number of center points.
__________________________________
HOW MANY CENTER POINTS?

(To ensure orthogonality and rotatability)
Choose α for rotatability; then...
Rule of Thumb:
No. of center points: nc = 4 ( n f + 1) − 2k
Uniform Precision:
If uniform precision is desired, find λ4 such that it is as

close as possible to the values in the Table below.
N
λ4 =
( F + 4F )
1
(4)
2
+4
where N=total number of design points.
# 2 3 4 5 6
Controls
λ4 0.7844 0.8385 0.8704 0.8918 0.9070
In many cases only near uniform precision designs can

be obtained (N is an integer).
__________________________________
CODING A CCD:
1) Find min, max for data and draw a line
design values: -α -1 0 1 α
-------------------------------------------------------------------
real values: 100 (200-∆) 200 (200+∆) 300
2) Specify a value of α (for example, α = 1.5):
Max − Center ∆
=
α 1
300 − 200
= 66 . 66 = ∆
1. 5
__________________________________
FINDING THE ‘BEST’ MODEL...
Suppose we want to find out the 'best' model after testing many subset
combinations.
WE NEED A DEFINITION FOR ‘BEST MODEL’
From the statistical point of view, the ‘best’ model is one that satisfies
the following conditions:
• It is parsimonious (i.e., has the minimum number of

parameters)
• Has the highest R2
• Has the lowest RMSE
It is a rare occurrence to find a model that satisfies all three conditions

in absolute terms.
=========> The best model is a trade-off solution.
In SAS, this is done in basically two steps:
1. Run PROC REG with the selection option turned on

to explore all the possible models.
2. Run PROC REG again for the 'best' model.
__________________________________
EXAMPLE (POLYMER)
Suppose we are trying to maximize the density (g/ml) of

a polymer which is a function of time and temperature.
The operating ranges are:
Time: 2 - 60 minutes
Temperature: 155 - 240 °C
A central composite design is a good choice.
__________________________________
THE DATA
Obs X1 X2 Y
1 60 190 101 Star point
5 30 240 . Star point: missing!
9 30 170 91 "Extra" point

We are missing an observation along the star axis, but

we have an "extra" point that perhaps we could use as
well. Notice that the three replicates of the center
point are placed last in the data set to allow SAS to
compute lack-of-fit statistics. The highlights of the
program will be described.
__________________________________
3/17/2005 10:25:53 AM
Results
Project : POLYMER EXAMPLE
Starting Response Model : Density (Y1)
Number of terms in the User Starting Model : 6

Model Terms : Intercept, X1, X2, X1X2, X1X1, X2X2
Weights : Unity


Labels
Bound Bound
Intercept -194.2872 62.1461 -346.3528 -42.2216
X2 Temperature 2.7479 0.6816 1.08 4.4159
X1X2 Time*Temperature 0.0073 0.0014 0.004 0.0106
X1X1 Time*Time -0.0163 0.0041 -0.0263 -0.0063
X2X2 Temperature*Temperature -0.0071 0.0018 -0.0116 -0.0026

Model 1811.2298 4 452.8075 22.4622 0.0009 Significant
X2 327.6181 1 327.6181 16.252 0.0069 Significant
X1X2 583.4636 1 583.4636 28.9436 0.0017 Significant
X1X1 323.2517 1 323.2517 16.0354 0.0071 Significant
X2X2 298.9047 1 298.9047 14.8276 0.0085 Significant
Residuals 120.952 6 20.1587
Lack of Fit 112.2853 4 28.0713 6.478 0.1382 No LOF
Pure Error 8.6667 2 4.3333
Replication Error 0 0 0
Repeats Error 8.6667 2 4.3333
Total 1932.1818 10
__________________________________

Observed Predicted Weight Standardized Residuals

Response Response s
Internal External
1 101 95.8243 1 5.1757 0.6855 2.0554 3.4496 1.8414 5.5785
2 72 76.7392 1 -4.7392 0.4207 -1.3868 -1.5358 0.2793 -1.4335
3 101 105.6436 1 -4.6436 0.6953 -1.8737 -2.6556 1.6025 -4.3948
4 70 67.142 1 2.858 0.6503 1.0765 1.094 0.4311 1.6344
5 70 70.6347 1 -0.6347 0.2899 -0.1678 -0.1535 0.0023 -0.1074
6 83 79.1178 1 3.8822 0.7009 1.581 1.8895 1.1714 3.1684
7 70 74.0579 1 -4.0579 0.5812 -1.3966 -1.5519 0.5414 -2.0028
8 91 90.073 1 0.927 0.2123 0.2326 0.2133 0.0029 0.1213
9 98 98.2558 1 -0.2558 0.2546 -0.066 -0.0603 0.0003 -0.0386
10 101 98.2558 1 2.7442 0.2546 0.7079 0.6751 0.0342 0.4322
11 97 98.2558 1 -1.2558 0.2546 -0.324 -0.2984 0.0072 -0.191
__________________________________
What is the CP statistic?
The SAS code above produces the output shown on the

following page. CP corresponds to the Mallows
statistic, C(p), which is computed as follows:
SSE ( p )
C( p ) = − (n − 2 p) + 1
MSE
where
MSE = error mean square for the full model

SSE(p) = error sum of squares for the subset model
containing p independent variables
n = total sample size
C(p) is a measure of total square error for a subset

model consisting of p independent variables (excluding
the intercept). It provides a good indication of how
many variables we should include in the model.
Mallows recommends that C(p) be plotted

against p and select that subset size where
the minimum C(p) approaches p+1,
starting from the full model
__________________________________
CCD Example
Best Subset Regression models using REG
All Possible Models Regression (using original data)
Model Selection Method : Best Subset Model

Subset Selection Algorithm : Leaps and Bounds
Selection Criterion : MSE
Model
R^2 C(p) MSE Rank Model
Terms
1 0 85.0068 193.2182 22 Intercept
2 0.265 62.7487 157.8045 20 Intercept, X1
2 0.3471 55.8478 140.1685 18 Intercept, X2
2 0.4274 49.1054 122.9378 17 Intercept, X1X2
2 0.1842 69.5329 175.1419 21 Intercept, X1X1
2 0.3117 58.82 147.7641 19 Intercept, X2X2
3 0.6105 33.7214 94.0753 15 Intercept, X1, X2
3 0.6289 32.1709 89.6176 14 Intercept, X1, X1X2
3 0.6304 32.0509 89.2726 13 Intercept, X2, X1X2
3 0.7316 23.5506 64.8338 7 Intercept, X1X2, X1X1
3 0.6035 34.3108 95.7698 16 Intercept, X1X2, X2X2
4 0.7518 21.8478 68.501 9 Intercept, X1, X2, X1X1
4 0.7583 21.3073 66.725 8 Intercept, X1, X1X2, X1X1
4 0.7827 19.2544 59.9795 4 Intercept, X2, X1X2, X1X1
4 0.7701 20.3129 63.4577 5 Intercept, X2, X1X2, X2X2
4 0.7678 20.5028 64.0814 6 Intercept, X1X2, X1X1, X2X2
5 0.785 19.0653 69.2511 10 Intercept, X1, X2, X1X2, X1X1
5 0.7679 20.4966 74.7381 12 Intercept, X1, X1X2, X1X1, X2X2
5 0.9374 6.2587 20.1587 1 Intercept, X2, X1X2, X1X1, X2X2
6 0.9405 6 23.0003 2 Intercept, X1, X2, X1X2, X1X1, X2X2
__________________________________
CCD Example
55
50
45
C(p), Mallows Statistics,
40
# of Parameters
# of Parameters
35
30 C(p)
25
20 Linear (# of
Parameters)
15
10
5
0
0 1 2 3 4 5
# of Regressors in the Model
Project Name : POLYMER EXAMPLE

Plot Name : Variable Importance Plot for Density(Y1) [RS Model 1]
3/17/2005 10:37:33 AM
Temperature 5.3612
Temperature * Temperature -5.1251

Model Terms
Time * Temperature 1.8718
Time * Time -1.3592
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
Standardized Estimates
__________________________________
Project Name : POLYMER EXAMPLE

Plot Name : User Plot1 [RS Model 1]
Response Variable: Density(Y1)
3/17/2005 10:37:59 AM
105.9
100.7
95.6
90.4
85.3
80.1
74.9
69.8
64.6
59.4
Time(X1): 21.8474
Temperature(X2): 179.7679
Predicted Density(Y1): 90.9542
__________________________________
NonLinear Models Nlin-1
SECTION V
Mechanistic Model Building

© Air Products and Chemicals, Inc.
Table of Contents
1: Introduction
Mechanistic Modeling
Mathematical Models- Classification based on Structure
2: Non-linear Regression
Comparison with Linear Regression
Regression Methodology and Output
Multi-Response Regression
Examples
3: Non-linear Design Strategies

D-Optimal Design
Sequential Design
Example
4: APPENDIX
Data Analysis using Excel

• Introduction
Objective: Develop fundamental understanding on the influence
of various control variables on desired responses.
Design Based
Model Building
Identify and characterize a fundamental relationship based

on physical/chemical understanding.
Control variables
Mechanistic Experimental
Description Design
(Model Form)
Regression
Design Based Analysis
Mechanistic Modeling
Model
Validation
Application Areas
• Kinetics, reaction rate expressions,
• Thermodynamic correlations,
• Transport correlations,
• Financial,
• Econometric,
• Business, or
• Engineering
Examples:
• Kinetic data analysis for Langmuir-Hinshelwood-Hougen-
Watson type of rate expressions or multiple uses of the
Arrhenius rate expression
r = ki C αA CBβ CCγ ... − k j CDδ CEλ ...
abx
r =
1 + bx
• Adsorption isotherm characterization, e.g. Dual-Site-
Langmuir isotherm. Equation of state or thermodynamic
characterization of multi-component mixtures.Growth rate
models etc.

Design-Based Mechanistic Modeling

Given: Structure of a state model:
f (U , X ,θ ) = 0
- Equations describing the system (process/product).
Determine: Unknown system parameters θ, so as to fully

characterize the system.
Approach:
• Step 1 – Identify appropriate responses that can be measured in the lab
and have enough sensitivity w.r.t. parameters.
• Step 2 – Define the Response Models

g (Y , U ,θ ) = 0
relating the response variables to state variables.
• Step 3 – Identify experimental design and collect data.
• Step 4 – Perform data regression to estimate the system parameters.
• Step 5 – Test for model adequacy and reliability.
• Step 6 – Sequential experimental design for model improvement

Mathematical Models
"A mathematical model is neither a hypothesis nor a theory. Unlike

scientific hypotheses, a model is not verifiable directly by an experiment.
For all models are both true and false.... The validation of a model is not
that it is "true" but that it generates good testable hypotheses relevant to
important problems. " (R. Levins, Am. Scientist 54:421-31, 1966)
• A mathematical model is a simple description of a

physical, chemical or biological state or process.
• Model helps researcher in designing better experiments
and making better sense of the results.
These fundamental models typically fall into two classes based

on their relationship with unknown variables,
General Linear Models
Relationships that are linear in parameters
General Non-Linear Models
• This class of models generally requires more effort and

knowledge in developing appropriate models than general
linear models.
• They can provide the greatest rewards, in that they have
greater flexibility in describing the system.

Distinction between General Linear and General

Non-Linear Models
a) General Linear Models:
Linear in the parameters/coefficients
y = b0 + b1 z1 + b2 z 2 + L + b p z p
y = ∑ bi z i
i
Where, zi = fi(x), where each fi can involve known constants;

none, one, or more xj; but no parameters from b
x = (x1, x2, ..., xm) = predictor or independent variables

(often the zi are called independent variables too)!
e.g.,
y = b0 + b1 x1 + b2 x12
y = b1 x1 + b2 log( x 2 − 3.8 x1 ) 0.5
Key distinction: A general linear model, as illustrated above,

often provides a response, which is non-linear, but does so with
a model form which is linear in the parameters.

b) Non-Linear Models:
Non-linear in parameters
e.g.,
− (θ1 +θ 2 t ) 2
y=e
θ1
y=
θ1 − θ 2
[e −θ t − e −θ t ]
2 1
ax1
y=
(1 + bx1 + cx 2 ) (Langmuir - Hinshelwood)
⎛ E ⎞
− ⎜⎜ ⎟⎟
⋅
y = k0 ⋅ e ⎝ 1 . 987 Temp ⎠
(Arrhenius)
y = W a X bZ c
y = β 1e − β 2 x1

Linearization of Nonlinear Models
Non-linear models which can be linearized into a general linear

model form via transformations are: Intrinsically Linear.
For example, to linearize
y = b1e − b2 x1
Take the log:
log y = b1 + b2 x Æ Linear Model
Models for which no transformations exist to put them into

general linear form are Intrinsically Non-Linear:
b1 x1
y=
(1 + b2 x1 + b3 x2 )

Classification Based on Structure

Algebraic Models – used to describe the steady-state
behavior of lumped parameter systems.
f (U , X ,θ ) = 0
Examples:
• Continuous Stirred Tank Reactors (CSTRs)
• Phase equilibrium in single/multi-stage flash calculations
• Chemical equilibrium
Ordinary Differential Equation Models– used to describe

the transient behavior of lumped parameter systems
f (U ' , U , X ,θ ) = 0 t = 0 U = U0
Examples:
• Start-up of CSTRs, batch or semi-batch reactors
• Cell growth models
• Micro-kinetic models
Partial Differential Equation Models – used to describe

the behavior of distributed parameter systems
f (U ' ,U xx ,U x , U , X ,θ ) = 0
Initial and Boundary conditions
Example-
• Diffusion/reaction in fixed bed reactor, monolith, etc.

Generalized Regression Model

State Equations
f (U ' , U , X ,θ ) = 0 t = 0 U = U0
Response Equations
g (Y , U ,θ ) = 0
Where
U = {U1, ….. Um}, a vector of state variables(dependent
variables)
Y = {Y1, ….. Yn}, a vector of measured responses,
X = {X1, ….. Xk}, a vector of independent variables
θ = {θ1, ….., θp}, a vector of parameters
f = {f1, ….. fm}, a vector of state equations
g = {g1, ….. gn}, a vector of response equations.
Note: In the above description state and model equations are in
the implicit form.
In many situations,
• State variables are explicitly defined in terms of
control variables,
• All the state variables are measured, so
Y =U
In those situations, regression model reduces to
Y = f ( X ,θ )

Regression Example
Three component kinetic system
A chemist is studying a system in which a reactant A
decomposes to form the desired product B, which in turn
decomposes into and undesired by-product C according to the
scheme:
First-order kinetics,
dC A
= −k C A
dt 1
dC B
= −k CB + k C A
dt 2 1
dCC
= k CB
dt 2
Where:
Y1 = U1 = CA is the concentration of A
Y2 = U2 = CB is the concentration of B
Y3 = U3 = CC is the concentration of C
Objective – Estimate the rate constants k1 and k2

Basic Regression Concepts
Linear Model (model linearly dependent on parameters)
True Model: y = β 0 + β1 x1 + ... + β n x n + ε
POPULATION
SAMPLE
Best estimate: y = b0 + b1 x1 + ...+ bn x n

NOMENCLATURE
INDEPENDENT VARIABLES
(deterministic or random variables
with constant variance)
y = β 0 + β1 x1 +... +β n x n + ε
Parameters
(Random variables)
ERROR TERM
DEPENDENT (random variable)
VARIABLE OR
RESPONSE
(Random variable)

Basic Assumptions: Linear regression
We assume there is a ‘true’ model (population):
y = β 0 + β 1 x1 + β 2 x 2 + ... + β n x n + ε
We estimate β 0 , β 1 ,...β n using b0 , b1 ,..., bn
Estimated Model:
yˆ = b0 + b1 x + b2 x 2 + ... + bn x n
Notice that E(ε)=0 (Expected value or average for error is zero)
We need to find a method to estimate the coefficients based on

some ‘goodness’ (Best fit) criteria.

Linear vs. Non-Linear Regression
In nonlinear regression, unlike linear regression, we usually

assume the functional relation between the response and the
factors has been derived from theory.
Linear Non-Linear
Independent of the initial guess Solution depends on the

Initial guess
Often "exact" statistical results Have to settle with

approximate statistical results
Unique least squares solution if May or may not have a

design matrix is full rank unique least squares solution
Can always locate the unique Iterative search techniques

least squares solution only; may not find best (or
any!) least squares solution
best design best design

=f(model form, design space) =g(model form, design space,
true coefficient values)

Generalized Non-Linear Regression Model
Model for response
Y = f ( X ,θ ) + ε
Where
Y = {Y1, ….. Yn}, a vector of responses
X = {X1, ….. Xk}, a vector of independent variables
θ = {θ1, ….., θp}, a vector of parameters
ε = {ε1, ….. εn}, a vector of experimental errors
Possible Constraints
L .B . ≤ C ( X , Y , Θ ) ≤ U .B .
or
Expectation Function:
E ( Y ) = f ( X ,θ )

Assumptions
• The errors are additive to the true model. No errors are
made in determining the control variable vector ξ .
or
• The regression model is unbiased.
ε i ~ N (0, σ 2 )
The errors follow a normal distribution with mean 0 and
variance σ2. The scatter about the model does not change
as a function of where you are in the design space.
Residual
Predicted y
• The errors are uncorrelated and independent.
The size and direction of any error is not related to the size
or direction of any other error

Regression Methodology
Least Square Criterion for Regression
Error Sum of Squares for the Non-Linear Model:
N exp
S (θ ) = ∑ {Yu − f ( X u ,θ )}2
obs
u =1
---(Objective Function)
Where
Nexp is the number of observations.
Least Square Estimate of θ minimizes S(θ).
Sum of vertical distances squared

about the regression line is
minimized in ordinary least squares

Regression Algorithm
Linearization of the Model
Let
θ 0 = (θ10 ,θ 20 ,K ,θ p 0 )
be an initial guess (starting point)
The first order TAYLOR SERIES expansion of f (ξ u , θ ) about

θ0 is,
⎡ ∂f (ξ u ,θ ) ⎤
p
f (ξ u ,θ ) = f (ξ u ,θ 0 ) + ∑ ⎢ ⎥ (θ i − θ i 0 )
i =1 ⎣ ∂θ i ⎦θ =θ
0
If we set
f u0 = f (ξ u ,θ 0 )
β i0 = (θ i − θ i 0 )
⎡ ∂f (ξ u ,θ ) ⎤
Z =⎢
0
⎥ ( Jacobian)
iu
⎣ ∂θ i ⎦θ =θ0

We have,
p
Yu − f u0 = ∑ β i0 Z iu0 + ε u
i =1
LINEAR FORM!!
Substituting this linear form in the objective function and

∂S
making ∂ θ = 0 we get the least square estimate of the
parameter:
θˆ = ( Z ′Z ) −1 Z ′y
Numerical Algorithms:
• Quasi-Newton (Gauss-Newton, Steepest Decent,…)

• Levenberg-Marquardt (Well-conditioned Newton Variant)
• Trust-region methods
• Optimization based methods, such as Sequential Quadratic
Programming (SQP) etc.
Due to non-linear nature of the problem, these algorithms
are iterative in nature and they do not guarantee a global
minimum.

Performance of such algorithms depends on
• Sophistication of the software and built in heuristics,

• Initial parameter guesses, and
• Form of Non Linear model.
General Concerns:
• Converge slowly
• Oscillate
• Diverge
NOTE:
Typically the trust-region based Marquardt algorithm is the

best choice by a wide margin. It is actually a cleverly weighted
combination of the steepest descent and the Gauss-Newton
algorithms. It takes advantage of the fact that typically steepest
descent works better than Gauss-Newton when one is far from
the solution and that typically Gauss-Newton works better than
steepest descent when one is near the solution.
For DAE models, a feasible path approach is utilized

which solves Differential-Algebraic model equations at
every step of the minimization.

Regression Output
Diagnostic Plots
• Parity plots: Observed Response Vs. Predicted Response.
• Residual plots: Look for non-random behavior, time-order

dependencies, non-constant variance
Converged Parameter Estimates
Test, if you can reach the same converged solution from

different initial guesses.
Parameter Confidence Interval

• Reliability of the estimates can be interpreted from the
confidence intervals. Wider the intervals, less reliable are
the estimates, which will lead to unreliable predictions.If
the confidence interval for a given parameter contains zero,
then parameter is insignificant or not estimable from the
given data set. You may want to run additional
experimental points or review the model form or start from
a different initial guess.
Model ANOVA for each response
Tests for model significance.
Lack-of-Fit ANOVA

Tests for model sufficiency.
Parameter correlation matrix

Gives an indication how correlated the parameters are. The
higher the correlation, more difficult it is to estimate those
parameters.
Variance-covariance matrix
Indicates the hidden dependencies between various Y responses.
Prediction confidence intervals

Indicates the predictive capabilities of the model. The tighter the
confidence intervals -- more reliable is the model.

Violations of Error Structure Assumptions
Regression software will provide parameter estimates whether

the error assumptions were met or not. If they’re not met, then
the model is estimated in a sub-optimal manner. Some common
violations and their fixes are:
• ε i ~ N (0, σ i 2 ) Î Error variance not constant
If the magnitude of the errors are related to where one is in

the design space or the size of the response, then weighted
least squares (WLS) analysis may be required (covered).
• The errors are not independent.

Time series analysis methods may be required (not covered).
• The control variable vector X is subject to error.

Generalized least squares methods may be required (not
covered)
If the violation of the assumptions is relatively small in nature,

then one can typically “get away” with using OLS.

Weighted Least Squares(WLS)
N exp
min ∑ j j
w ( y obs
− y pred 2
j )
Θ j =1
L .B . ≤ C ( X , Y , Θ ) ≤ U .B .
Where wj are the regression weights.
Weights:
• Equal – (Ordinary) Least Squares
• 1/sqr(Y) – Approximate Relative Error Criteria for Y>0
• 1/Y – Requires Y>0 and Faster Than Relative Fanning
WLS is useful when the assumption of equal-variance is not

satisfied.
Residual
Predicted y

Transformations
Possible reasons for the use of transformations:
• To better meet the assumptions of the intended Ordinary

Least Squares or Weighted Least Squares analysis.
• To implicitly change the shape of the model in the hopes of

improving how well the model can mimic the data.
• To linearize the model such that General Linear Models

software can provide starting estimates for the non-linear
regression analysis.
• To obtain a better scaling such that the non-linear regression

software can converge (or can converge more rapidly).
Transformations can be either beneficial or detrimental!

Transformations and error structure
The use of transformations to linearize a non-linear model into

the general linear form dramatically impacts the error model.
β
Ex: Use of log transformation to linearize: y = x
Case 1: Additive error in original metric (assumed):
y = x β + ε i becomes log y = log( x β + ε i )
==> Errors in log y are not additive!
Case 2: Multiplicative error in original metric (assumed):
y = x β ε i becomes log y = β log x + log ε i
==> Errors in log y are additive.
If in Case 1 fit the model in the original metric.

If in Case 2 fit the model in the transformed metric.

Transformations and error structure (cont)
Ex: Enzyme Kinetic Model
θx
y= + εi
µ+x
Inverse transformation leads to
1 1 µ 1
= + • + εi
*
y θ θ x
Error model is still additive in this situation, but the

transformation does not preserve normality (e.g., if ε i follows a
normal distribution then ε i does not and vice-versa.
*
Model fitting should take place in the metric which provides

normality (providing other assumptions are met as well).
Normality is a second level concern after model fit.

Computer Packages:
1. Statistical Studio
2. Athena
3. SimuSolv
4. SAS
5. STATISTICA
6. SIGMAPLOT
7. Delta Graph
8. NAG ROUTINES
9. EXCEL (solver) – see appendix A

Practical Considerations
• Initial Parameter Estimates: Start from different sets

of values to ensure that you have found the global
optimum.
• Dependencies in responses and/or residuals.

Be aware of
Mass balances
Imputed responses
>> Program checks for these dependencies.
• Check for extreme multi-collinearities between

estimated coefficients. Correlation matrix is
provided.
>> Model is possibly over-parameterized.
• For some problems, to improve convergence,

Parameter transformations to encourage linear behavior
Transform model and data.
• Missing data
Single Response: No problem
Multi-response: More difficult, may be handled in next
version.

Example 1
Computer Code in SAS
* Simulate nine observations of y from x using the model form Y = 2X0.7 ;

* with random error (normally distributed with mean 0 and standard deviation .2) ;
* added to each y value. ;
data one;
do x = 2 to 10 by 1;
y = 2*(x**.7);
y = y + 0.2*rannor(5934491);
output;
end;
* Specify/perform the nonlinear regression and write out a data set with predicted ;
* and confidence limits. ;
proc nlin method = Marquardt maxiter = 50 data = one;

parameters a=.5 b=.5;
model y = a* (x* *b);
der.a = x**b;
der.b = a*(x**b)*Iog(x);
output out=two p=pred_y 195m=195m u95m=u95m 195=195i u95=u95i;
title 'Nonlinear Regression Analysis Example';
* Print the the data set written out by the nlin procedure.
proc print data= two;

id x
var y pred_y 195m u95m 195i u95i;
run;

Resulting Record of Search
NON-LINEAR LEAST SQUARES ITERATIVE PHASE
DEPENDENT VARIABLE: Y METHOD: MARQUARDT

ITERATION A B RESIDUAL SS
0 0.500000000 0.500000000 325.257854318

. 1.807528042 1.698003533 16963.107887309
. 1.781462131 1.713406244 17696.721025570
. 1.688711990 1.699328508 14575.426241539
. 1.321072109 1.345374852 997.784719132
1 0.703847879 0.710403187 194.401194264
2 1.134863702 1.003665806 5.219979785139
3 1.181401044 0.963339380 3.490039609482
4 1.529744417 0.818077584 1.488012178417
5 1.893381626 0.718583404 0.411324978797
6 1.943756470 0.717021366 0.162085754972
7 1.944000052 0.716996762 0.162082913317
8 1.944001278 0.716996443 0.162082913311
CONVERGENCE CRITERION MET.

Example 2
A chemist is studying a system in which a reactant A
decomposes to form the desired product B, which in
turn decomposes into and undesired by-product C
according to the scheme:
β1 β 2
A⇒ B ⇒ C
First-order kinetics,
dη
− 1 =β η
dx 1 1
dη
− 2 = β η −β η
dx 2 2 1 1
Where:
η1 is the concentration of A
η2 is the concentration of B

The concentration of B at any time x is given by
β1 −β 2 x −β1x
η2 = (e −e )
β1 − β2
Experimental Result:
Time at which reaction is Relative concentration of
terminated product B
x y
1 0.22
2 0.51
4 0.48
5 0.29
6 0.20
7 0.12
Used an iterative nonlinear least square program (CSS

Statistica©) to estimate β1 and β2.
Numerical method used was Gauss-Newton.

The NLIN Procedure

Dependent Variable y
Method: Gauss-Newton
Iterative Phase
Sum of
Iter beta1 beta2 Squares
0 1.0000 0.5000 0.1630

1 0.4603 0.4166 0.0499
2 0.5119 0.3386 0.0366
3 0.5171 0.3479 0.0363
4 0.5149 0.3475 0.0363
5 0.5154 0.3475 0.0363
6 0.5153 0.3475 0.0363
7 0.5153 0.3475 0.0363
8 0.5153 0.3475 0.0363
NOTE: Convergence criterion met.
Estimation Summary
Method Gauss-Newton
Iterations 8
Subiterations 1
Objective 0.036312
Observations Read 6
Observations Used 6
Observations Missing 0
The NLIN Procedure
Sum of Mean Approx

Source DF Squares Square F Value Pr > F
Model 2 0.6411 0.3205 35.31 0.0029

Error 4 0.0363 0.00908
Uncorr Total 6 0.6774

Approx
Parameter Estimate Std Error Approx 95% Confidence Limits
beta1 0.5153 0.1374 0.1339 0.8968
beta2 0.3475 0.0549 0.1951 0.4999
Nonlinear modeling of chemical reaction

y=((.515334)/((.515334)-(.347521)))*(exp(-(.347521)*x)-exp(-(.515334)*x))
0.5
y, conc. of product B
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6 7 8
x, time at which reaction is terminated

Non-Linear Design Strategies
Why Statistical Design of Experiments?

The least squares solution of the linearized model
y = Zβ + ε
is
b = ( Z ' Z ) −1 Z ' y
But the vector of coefficients, b, is a normal random variable

with
Mean = E (b) = β
Variance = σ b = σ ( Z ' Z ) or, approximately,

2 2 −1
s 2 (b) = MSE ( Z ' Z ) −1
σ b2

The matrix Z ' Z is called the information matrix

Variance of coefficients:
^
s ( β ) = MSE ( X ′X ) −1
2
Depends upon the Depends upon the

experimental set-up, experimental design
procedure, skills, used for modeling.
instrumentation.
Minimize for best modeling results !

==> Statistical experimental designs maximize the model
precision through the design matrix.

Experimental design for Nonlinear Models
For a general linear model an “optimal” experimental

design is a function of:
• the model’s form

• the selected design space
• the selected number of experiments
For a general non-linear model an “optimal” experimental

design is a function of:
• the model’s form

• the selected design space
• the selected number of experiments
and
• the true coefficient values
But the true coefficients in the model are the very thing we’re
trying to estimate!

Design Criteria for Non-Linear Models

(“Alphabet Optimality”)
Choose the experimentation points to maximize the

volume of the inference region.
D-Optimality : Maximize the determinant of the

information matrix.
T
Maximize Z Z
Invariant to non-singular recoding of the design matrix.
Efficient Method for computer implementation
A-Optimality : Maximize the determinant of the

information matrix.
T
Maximize trace ( Z Z)
Variant to non-singular recoding of the design matrix.
Computationally expensive to implement.
I-Optimality : Minimize the average prediction

variance
T T −1
Minimize z ( Z Z ) z over the candidate space.

D-Optimality
• In general linear models the D-optimality criterion is

accomplished by:
Maximizing X ' X , where X is the design matrix
The criteria minimizes the uncertainty in the estimation of the

overall model within the constraints of the design region and
the number of experiments selected.
• In general non-linear models the D-optimality criterion is

accomplished by:
Maximizing Z 0 ' Z 0 , where Z 0 is the nxp Jacobian matrix

at β = β 0
The criteria minimizes the uncertainty in the estimation of the

overall model within the constraints of the design region, the
number of experiments selected, the quality of the parameter
estimates β0, and the reasonableness of the use of first order
derivatives in the neighborhood of β0.
Note: it is an approximate design criterion!

Design Algorithms
Grid Search Methods:

Search is done on a discretized design space.
• Failure less likely
• Computationally much simpler
• Search bounds handled trivially
• Must search enough points
• High dimensionality implies coarser grids
Popular Methods
• Sequential or Dykstra method
• Simple exchange (Wynn-Mitchell) method
• DETMAX algorithm (exchange with excursions)
• Modified Fedorov (simultaneous switching).
None of the algorithms guarantee the Optimal

solution.
Gradient-Based Search Methods

Search is done on a continuous domain based on the local
gradient information
Continuous Search for Next Point
• Failure much more likely
• Computationally extremely intensive
• Handling of search bounds can be complex

D-optimality and non-linear models

D-optimality is an extremely focused strategy, it assumes that
the true model form is KNOWN. This implies:
• One can’t select all design points with this criterion

• Need to add points to check model lack-of-fit
• Strategy will often want to run the same small set of best
points again and again
The entrance of the parameter estimates into the solution for the
optimal design points indicates that the use of
Stagewise/sequential design
can be very beneficial in non-linear problems.

Sequential Design
Objectives:
• Improve precision of estimates of some or all of the

parameters.
Remember, joint inference region for the parameters has volume
T
proportional to Z Z
• Model Selection and Discrimination.
• Optimal estimation of functions, e.g. operating states
or profit functions.
Approach:
• Use D-optimal Design to select additional points.
Advantages:
• Uses already existing data set.

• Allows focus on subset of parameter space.
• Can check on model validity
• Can use estimates from previous runs.

Example 3: Isomerization of Bicyclo Hexane
From page 287, Applied Parameter Estimation for Chemical Engineers

The following non-linear model was proposed to for the
fraction of original material remaining (pseudo-conv.), y,
as a function of time, x1, and temperature, x2.
Y = exp(-p1*x1*exp(-p2*(1/x2-1/620)))
Design Space of Interest
Variable Lower Bound Upper Bound

Time 0 150
Temperature 600 640
The Objective is to identify this model (estimate

parameters p1 and p2) using Design-Based Modeling
Approach. To begin with, the initial guesses of the
parameters is given to be: (p1 = .001, p2 = 10000)
Actual Solution: p1 = 0.00373, p2 = 27643

10 Experiments
Expt. No. Time Temperature Conversion
1 10 600 0.991906293
2 40 600 0.967431911
3 80 600 0.934848257
4 100 600 0.91931483
5 150 600 0.881679952
6 10 640 0.860220306
7 40 640 0.548607927
8 80 640 0.301761912
9 100 640 0.222038881
10 150 640 0.106088191
Project Name : Isomerization of Bicyclo Hexane

Regression Analysis :Results
Dependent Variable Conversion
Method: Gauss-Newton
Iterative Phase
Sum of
Iter p1 p2 Squares
0 0.00100 10000.0 1.3426

1 0.00335 45785.0 0.2160
2 0.00484 19231.6 0.0346
3 0.00398 25945.1 0.000871
4 0.00373 27571.9 6.158E-6
5 0.00372 27676.0 2.569E-6
6 0.00372 27676.3 2.569E-6
NOTE: Convergence criterion met.
Estimation Summary
Method Gauss-Newton
Iterations 6
R 1.088E-6
PPC 1.57E-9
RPC(p2) 0.000013
Object 0.000011
Objective 2.569E-6

Observations Read 10
Observations Used 10
Observations Missing 0
Sum of Mean Approx

Source DF Squares Square F Value Pr > F
Model 2 5.6088 2.8044 8732760 <.0001

Error 8 2.569E-6 3.211E-7
Uncorrected Total 10 5.6088
Approx
Parameter Estimate Std Error Approximate 95% Confidence Limits
p1 0.00372 6.877E-6 0.00370 0.00373
p2 27676.3 36.5567 27592.0 27760.6
Approximate Correlation Matrix

p1 p2
p1 1.0000000 -0.8888459
p2 -0.8888459 1.0000000


Task Name : One-at-a-Time Approach
Plot Name : Prediction Plot for Conversion(Y1)
Model : exp(-p1*x1*exp(-p2*(1/x2-1/620)))
03/13/2001 9:26:47 AM
Response
1.0
0.9
0.8
0.7
Predicted Response
Observed Response
0.6
0.5
0.4
0.3
0.2
0.1
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160

Tim e(X1)
P r o j e c t N a m e : I so m e r i z a ti o n o f B i c y c l o H e x a n e
T a sk N a m e : A u to m a te d A p p r o a c h
P l o t N a m e : U se r P l o t1
0 3 / 1 3 / 2 0 0 1 9 :4 7 :4 0 A M
1.000
0.919
0.837
0.756
0.675
0.593
0.512
0.431
0.349
0.268
0.187
0.105
T im e ( X 1 ) : 5 . 0 3 7 9 6 4 e + 0 0 1
T e m p e r a tu r e ( X 2 ) : 6 .1 6 2 4 5 6 e + 0 0 2
Pr e d ic t e d R e s p o n s e ( Y 1 ) : 8 .6 6 5 5 3 3 e - 0 0 1


Task Name : Automated Approach
03/13/2001 10:04:59 AM
1.000
0.919
0.837
0.756
0.675
0.593
0.512
0.431
0.349
0.268
0.187
0.105
Time(X1):
Temperature(X2):
Predicted Response (Y1):

Model Validation
Model validation checks for the performance of the model
in the regions of the design space,
O Which were left out by D-optimal design
O Which show “Unusual response”, e.g. model
discontinuities etc.
D-optimal points
Validation points
Design Space
Validates the model form throughout the design
space.

Nature of D-optimality
• D-optimality can be used to pick stages as small as

one experiment.
• Ideally suited for selection of one to several next

experiments and iterating until the resulting model
has the desired reliability.
• Can be as dynamic a design process as is practical.
• If experimental results are obtained “simultaneously”

with the end of the experiment and analysis is
“automated” a stage size as small as one may be
practical.
• When greater time lags and/or analysis effort are

required consider stages with several additional
experiments.

Robustness of D-optimality
The “Good Side” of the Force
Robustness is as narrow and focused as the target (tightest fit for
specific model in minimum # experiments).
Extremely robust if the model form is correct or at a minimum

includes the correct model form as a subset of its parameter
values. Nonlinear modeling case also assumes suitability of
linearization about solution.
Current commercial software also implicitly assumes the

suitability of ordinary least squares.
However …
All models are wrong, some models are useful (Box)
The “Dark Side” of the Force

For models which are not useful enough, the laser-like focus on
the specific model form proves to be too narrow.
D-optimality collects no data to check the appropriateness of the
model form under investigation (model lack-of-fit).
The experiments selected solely by D-optimality will not even
permit estimation of many alternative model forms.
Use of the wrong error criterion can negatively impact point
selection.
In the nonlinear case, poor initial guesses of parameter values
can damage point selection if program is built without iteration.

D-optimality in Commercial Software

• Better statistical packages and most experimental
design packages provide D-optimal design tools for
linear models.
• Non-linear D-optimal design tools almost nonexistent
and often fragmentary in nature.
• Even the linear model based routines can be limited
in their searching abilities.
• Hidden assumption of subsequent use of “ordinary”
least squares haunts existing software.
• Tools typically provide strict textbook D-optimality
(typically an overly narrow design focus).
• Guidance on proper tool usage is generally weak.

REFERENCES
Box, G.E.P., and Lucas, H.L., 1959. Design of experiments in non-linear
situations. Biometrika 46, 77-90.
Bates, D.M., and Watts, D.G., 1988, Nonlinear Regression Analysis and its
Applications, John Wiley and Sons, Inc., New York.
Englezos, P., and Kalogerakis, N., 2001, Applied Parameter Estimation for
Chemical Engineers, Marcel Dekker, New York.
Atkinson, A.C., and Hunter, W.G., 1968. The design of experiments for parameter
estimation. Technometrics, 10, 271-289.
Draper, N.R., and Smith, H., 1981. Applied Regression Analysis, Second Edition.
John Wiley and Sons, Inc., New York.
Johnson, M.E., and Nachtsheim, C.J., 1983. Some guidelines for constructing
exact D-optimal design on convex design spaces. Technometrics, 25, 271-277.
Box, M.J., and Draper, N.R., 1971. Factorial Designs, the |X'X|
criterion, and some related matters. Technometrics, 13, 731-742.
Dykstra, O. 1971. The augmentation of experimental data to maximize |X'X|.

Technometrics, 13, 682-688.
Hamilton, D.C., 1982. Examples and computational aspects of second order

designs for non-linear regression parameters.
Buzzi-Ferraris, G., and Forzatti, P. 1983. A new sequential experimental design

procedure for discriminating among rival models. Chem. Engr. Sci., 38, 225-232

APPENDIX: DATA ANALYSIS USING EXCEL
A key step in kinetic modeling is to obtain the estimates and confidence intervals of kinetic
/thermodynamic parameters by minimizing the deviations of the data points from the model predicted
responses. Depending upon the reaction mechanisms for the system under consideration, type of
reactor system used and nature of response(s) measured, proposed models can be written as algebraic
equations or as a system of differential equations. Furthermore, these models can have linear or
nonlinear dependencies on the unknown parameters. For linear models, Analysis Toolpak, an Add-in
package in Excel can provide the best fit to a set of data points and calculate basic regression statistics
(see Appendix A). In kinetic studies models often have nonlinear dependencies on parameters. In
addition there are thermodynamic and positivity constraints on some or all of the parameters. Some
common types of kinetic models are:
1. Integrated Rate models for homogeneous systems, e.g. initial rate measurements for various
feed concentrations and or temperature,
r = ki CαA CBβ CCγ ... − k j CDδ CEλ ...
2. Differential Models for the concentration : Time dependent studies where species
concentrations are measured as a function of time. Changing concentrations are described by a
system of differential equations
dC
= f (C, t, Θ)
dt
3. Langmuir-Hinshelwood-Hougen-Watson models in heterogeneous catalysis. These models
have highly nonlinear algebraic rate expressions with equality and/or inequality constraints. The
simplest example of this type is:
abx
y=
1 + bx
Model 3 can be put into general linear form through use of the reciprocal transformation (some, but not
all nonlinear models can be put into general linear model form via transformations).
1 1 1
= +
y abx a
Now the Linest function in Excel can be used to provide the interval estimates for the coefficients
(1/ab) and 1/a, which can then be used to obtain the point estimates of the original parameters a and b.
Statistical issues arising from estimation in transformed metrics is discussed on pages 13-15. For
example, the standard errors of a and b may need to be calculated by accounting for the propagation of
error relationships during the transformation, a nontrivial exercise. When such situations arise it is
advisable to treat the model in its original form and perform nonlinear regression analysis.

Using the Solver in Excel
The solver is an optimization package that finds a maximum, minimum or specified value of a
Target cell by varying the values in one or several Manipulating cells. Solver allows for the addition
of equality, inequality or integer constraints. This serves as a powerful tool for solving a wide variety
of problems including parameter estimation, linear, nonlinear, and integer optimization.
The solver is an add-in and can be selected from the Tools menu. To use the solver to perform
nonlinear parameter estimation follow the procedure given below. The procedure is demonstrated in
next section :
1. Data Tabulation : Start with a worksheet containing the data, the independent variable(s) , x
and observed dependent variable(s), yobs to be fitted.
2. Model Prediction : Assign unknown parameters to certain manipulating cells. To start, guess
some values for these parameters. Provide the formula for the model to predict the response(s),
ypred for each experimental data point. This step is straightforward for algebraic models but for
differential models, predictions will require additional calculations for the solution of differential
equations, see Example 4.
3. Calculate Residuals : For each data point tabulate the residuals (ypred -yobs).
4. Formulate Objective function : This step is essentially related to the choice of the objective
function (O.F.) for regression, e.g. ordinary least squares (OLS) or weighted least squares(WLS) or
any other liklihood based function to be minimized. OLS is the simplest and most commonly used
and O.F. in this case is sum of squares of the residuals, After making the choice, simply provide the
expression for O.F. in a target cell.
5. Use Solver : Load the solver from the Tools menu and minimize the O.F. in the target cell by
changing the parameters in the manipulating cells, see Fig. below. Minimization is a repetitive
procedure and the values of the cells will change during the course of minimization (McFedries,
1995). After finding the minimum, Excel generates Answer and Sensitivity reports.
6. Obtain Regression Statistics : Solver provides the parameter estimates but it doesn’t have any
means for obtaining standard errors and other useful model statistics. This issue can be partially

addressed by use of the macro SOLVSTAT.XLM provided by Billo (1997) (see Appendix B
References). This macro provides the standard errors (s.e.) of estimated parameters, s.e. in y and
the R2 statistic. In many simpler modeling situations this information along with analysis of
residuals is sufficient to complete the data analysis. For analyses requiring more detailed statistics
such as confidence intervals for model predictions either a commercial statistical package such as
SAS will be required or an even more sophisticated Excel macro would need to be located.
Solver provides user with a number of options for controlling the way the minimization is
performed, see Fig. below. The user can specify the maximum time or maximum number of iterations
during the solution procedure. Furthermore, it allows the user to select between Newton and conjugate
gradient methods for the search of minimum and also permits the automatic scaling of the equations.
The derivatives required during the minimization are calculated numerically either by forward or central
difference scheme. The most recent update of solver, known as Premium Solver Plus, which is
compatible with the Excel 95 provides greater speed and robustness. It is capable of solving Linear
problems up to 16000 variables and nonlinear problems up to 4000 variables and without any restriction
on the number of constraints. More information about the solver and its algorithm can be obtained from
site www.frontsys.com.
Test Problems
1. Negative Exponential Growth Model : This two parameter model along with the
associated data set is taken from SAS User’s guide. This model is used to describe ionic transport
through membranes :
y = a (1 − exp( −bx ))

x yobs ypresd (ypred-yobs) (ypred-yobs)2 Initial
20 0.57 0.565727 -0.00427 1.83E-05 Parameter
30 0.72 0.713224 -0.00678 4.59E-05 Estimates
40 0.81 0.810182 0.000182 3.32E-08 a = 0.9
50 0.87 0.873917 0.003917 1.53E-05 b = 0.3
60 0.91 0.915814 0.005814 3.38E-05
70 0.94 0.943354 0.003354 1.12E-05
80 0.95 0.961458 0.011458 0.000131
90 0.97 0.973358 0.003358 1.13E-05
100 0.98 0.981181 0.001181 1.39E-06
110 0.99 0.986323 -0.00368 1.35E-05
120 1 0.989704 -0.0103 0.000106
130 0.99 0.991926 0.001926 3.71E-06
140 0.99 0.993386 0.003386 1.15E-05
150 1 0.994347 -0.00565 3.2E-05
160 1 0.994978 -0.00502 2.52E-05
170 0.99 0.995393 0.005393 2.91E-05
180 1 0.995665 -0.00433 1.88E-05
190 1 0.995845 -0.00416 1.73E-05
200 0.99 0.995962 0.005962 3.56E-05
210 1 0.99604 -0.00396 1.57E-05
SS= 0.000577
Negative exponential growth curve Residual Plot
1.2 0.015
1
0.01
0.8
RESIDUALS
0.005
0.6
y
0
Measured
0.4 0 5 10 15 20 25
Predicted -0.005
0.2
-0.01
0
0 50 100 150 200 250 -0.015
x x
Excel worksheet with the plots showing fitted response and normality of residuals
Software Estimate of a s.e. of a Estimate of b s.e. of b

Excel 0.9961885659 0.0016137998 0.0419538871 0.000398229
SAS 0.9961885657 0.0016138001 0.0419538868 0.000398229
Estimated Parameters and Standard Errors : Comparison between Excel and SAS
2. Isomerization of Pentane (Hougen-Watson Kinetics)

Data on reaction rate of the catalytic isomerization of n-pentane to iso-pentane versus the partial
pressures of hydrogen(x1), n-pentane(x2), and iso-pentane(x3) are given by Bates and Watts (1988),
and highlighted below in the worksheet. A proposed model for these data with four parameters,
a,b,c, and d;
ac ( x 2 − x 3 / 1.632 )
f =
1 + bx1 + cx 2 + dx 3
P-Hydrogen P-Pentane P-Isopentane Msd. Rate Pred. Rate Residual Square Res. Initial
205.8 90.9 37.1 3.541 3.664623518 0.123624 0.01528277 Parameter
404.8 92.9 36.3 2.397 2.439770065 0.04277 0.00182928 Estimates
209.7 174.9 49.4 6.694 6.382446885 -0.31155 0.09706534 a = 30.0
401.6 187.2 44.9 4.722 4.91661242 0.194612 0.03787399 b = 0.1
224.9 92.7 116.3 0.593 0.728752532 0.135753 0.01842875 c = 0.05
402.6 102.2 128.9 0.268 0.572920716 0.304921 0.09297664 d = 0.5
212.7 186.9 134.4 2.797 3.108380767 0.311381 0.09695798
406.2 192.6 134.9 2.451 2.500413157 0.049413 0.00244166
133.3 140.8 87.6 3.196 3.884481591 0.688482 0.4740069
470.9 144.2 86.9 2.021 2.269038164 0.248038 0.06152293
300 68.3 81.7 0.896 0.642288292 -0.25371 0.06436963
301.6 214.6 101.7 5.084 4.348546959 -0.73545 0.54089118
297.3 142.2 10.5 5.686 6.305280984 0.619281 0.38350894
314 146.7 157.1 1.193 1.242021333 0.049021 0.00240309
305.7 142 86 2.648 2.855305461 0.207305 0.04297555
300.1 143.7 90.2 3.303 2.802950297 -0.50005 0.25004971
305.4 141.1 87.4 3.054 2.787346017 -0.26665 0.07110435
305.2 141.5 87 3.302 2.812240848 -0.48976 0.23986403
300.1 83 66.4 1.271 1.571592021 0.300592 0.09035556
106.6 209.6 33 11.648 11.67934651 0.031347 0.0009826
417.2 83.9 32.9 2.002 2.202588555 0.200589 0.04023577
251 294.4 41.5 9.604 9.898804916 0.294805 0.08690994
250.3 148 14.7 7.754 7.035909783 -0.71809 0.51565356
145.1 291 50.2 11.59 11.50758569 -0.08241 0.00679212
SS= 3.23448228
Excel Worksheet for Example 2
Software a s.e. of a b s.e. of b c s.e. of c d s.e. of d

Excel 35.92 8.21 0.071 0.1786 0.0377 0.1 0.167 0.415
SAS 35.92 8.21 0.071 0.1783 0.0377 0.0998 0.167 0.415
Estimated Parameters and standard errors :Comparisons between Excel and SAS

Reaction Rate (per hour)
14 14
12 12
10 Observed 10 Observed
Pedicted 8 Pedicted
8
6
6 4
4 2
2 0
0 0 50 100 150 200
0 200 400 600 Isopentane partial pressure (psia)
Hydrogen partial pressure (psia)
14
0.8
12 Observed 0.6
10 Pedicted 0.4
Residuals
8 0.2
6 0
-0.2 0 100 200 300 400
4
-0.4
2 -0.6
0 -0.8
0 100 200 300 400 -1
n-Pentane partial pressure (psia) n-Pentane partial pressure (psia)
Fitted responses vs. various predictor variables and the plot showing normality of residuals
Conclusions
We have demonstrated the use of Excel for nonlinear parameter estimation on a number of kinetic
problems. The results from the first two examples presented above show that the parameter estimates
and standard errors obtained using Excel matched very well with those from SAS. Our interest in using
Excel for nonlinear regression analysis is justified by its comparable performance, easy use, familiarity
and popularity of Excel throughout the company. For those situations which require model confidence
limit estimation, more advanced statistical tools than those currently available in Excel would be
required.
References
1. Bates, D.M., D.G. Watts, Nonlinear Regression Analysis and its Applications, John Wiley and
Sons, N.Y., 1988.
2. Berk, K.N., P.M. Carey, Data Analysis with Microsoft Excel, Doxbury Press, N.Y., 1998.
3. Billo, E.J., Excel for Chemists : A Comprehensive Guide, Wiley-VCH, New York, 1997.
4. McFedries, P., Excel for Windows 95 - Unleashed, Sams Publishing , Indianapolis, 1995.
5. SAS User’s Guide, Vol. 2, Version 6, SAS Institute, Carey, NC, 1990.

SECTION VI
Mixture Design Strategy:
An Overview
Mixture Design Mix-1
MIXTURE EXPERIMENTS
• A mixture experiment involves varying

the proportions of two or more
ingredients, called components of the
mixture
• Studying the changes that occur in the

measured properties (responses) of the
resulting end product
• Process variables or the total amount of

the mixture may also be studied in the
experiment
_________________________________
What we will learn...
• How to choose the proportion of

ingredients to be used in an experiment
• How to analyze the data collected
• How to interpret the results
• How to develop models
• Examples
• Which software to use
_________________________________
Factorial vs. Mixture
• Factorial Experiments:
⇒ Response varies depending on the actual

amounts of each of the independent
variables
⇒ Effect of temp and pressure on a product

yield
• Mixture Experiments:
⇒ Independent or controllable variables

represent proportionate amount of the
mixture (by volume, by weight, or by mole
fraction)
⇒ Effect of the different blends of water,

alcohol, and urea on the viscosity of a liquid
detergent
_________________________________
Mixture Experiments: Constraints
• Let q = # of components in the mixture
• xi = proportion of ith component then
0 ≤ xi ≤ 1, i = 1, 2, ..., q
q
∑ xi = x1 + x 2 + . . . + x q = 1. 0
i=1
_________________________________
Other Constraints
Single Factor:
a < x i < 1, 0 < x i < b

Multiple:
a < xi < b
Multifactor:
a < ( xi + x j ) < b
_________________________________
3 Factor Factorial and Mixture Design Spaces
Design Space Collapses
Triangular Region: x1 + x2 + x3 = 1
(0,0,1)
SIMPLEX
Coordinate System
x3
(0,1,0)
(0,0,0) x2
(1,0,0)
x1
_________________________________
Design Spaces
Two Variable Design Space

50% x1 100% x3
100% x1 50% x2 100% x2 50% x3

50% x2
100% x4
100% x1 100% x2
50% x4
50% x3
100% x3
100% x1
Four Variable Design Space

100% x2
_________________________________
3-Factor Mixture Design Space
Vertices represent 100% of a given

factor
x3 = 1
70% x3
x1=x3 33% each
x2 = 1 x1=0.5 x1 = 1
x2=0.5
_________________________________
3-Factor Constrained Mixture

Design Space
x1: 5-50%, x2: 50-85%, x3: 10-50%,
_________________________________
Objectives
• To study the effects of the ingredients on
some measurable response in an attempt
to find the blend or blends that produce
the optimal response or a set of optimal
responses
e.g., Minimize Viscosity of a liquid

detergent by varying the proportions
of Water, Alcohol, and Urea
• To model the blending surface with some

form of mathematical equation so that
predictions can be made
e.g., Linear, quadratic or cubic response

surface
_________________________________
Steps in Planning a Mixture Experiment
1) Define the objective of the experiment
2) Select mixture components (and any

other factors)
3) Identify any constraints on the mixture

components (this will fix the
experimental region)
4) Specify responses to be measured
5) Propose an appropriate model form
6) Select an experimental design to fit the

proposed model and test for model
adequacy
_________________________________
Steps after Experimentation
• Fit the model to data

• Test for model adequacy
• If the model is inadequate, find the best
model which fits the data
• Plot model
• Optimize response variable(s) for the
best blending composition
_________________________________
Simplex-Lattice Design
Simplex Lattice - A uniformly spaced set of points

on a simplex. This will provide, in general, the
best set of n experiments to accommodate an n term
model.
Definition - A {q,m} simplex lattice has

(m+1)equally spaced points between 0 and 1 for
each of q design factors i.e.,
1 2
xi = 0, , ,...,1
m m
q m # of points model
3 1 3 linear
3 2 6 quadratic
3 3 7 spec. cubic
3 3 10 cubic
4 1 4 linear
4 2 10 quadratic
4 3 14 spec. cubic
_________________________________
(3,2) Simplex Lattice
x1=1
(1,0,0)
(1/2,1/2,0) (1/2,0,1/2)
(0,1,0) (0,0,1)
x2=1 (0,1/2,1/2) x3=1
_________________________________
Simplex Lattices
(3,2) Simplex Lattice (3,2) Simplex Centroid Lattice
(3,3) Lattice (4,2) Lattice
_________________________________
Canonical Form of the Mixture Polynomials
A general form of regression function that

can be fitted to the data collected at the
points of a {q,m} simplex-lattice is called
the CANONICAL form of the polynomial
They are derived by imposing the

restriction:
q
∑ x i = x1 + x 2 + . . . + x q = 1. 0
i=1
to the terms in the standard polynomial
_________________________________
First Order Model
• Two Components, x1 and x2

• Factorial model:
y = β0 + β1 x1 + β2 x 2
Since (x1 + x2)=1 we can replace
bo with bo(x1 + x2)
y = ( β0 + β1 ) x1 + ( β0 + β2 ) x2
= β1′x1 + β2′x2
_________________________________
Second Order Model
• Two Components, x1 and x2

• Factorial model:
y = β0 + β1x1 + β2 x2 + β12 x1x2 + β11x12 + β22 x22
Replacing bo by bo(x1+x2), x12 by x1(1-x2),

and x22 by x2(1-x1), we obtain
y = ( β0 + β1 + β11 ) x1 + ( β0 + β2 + β22 ) x 2
+ ( β12 − β11 − β22 ) x1 x 2
= β1′x1 + β2′x 2 + β12′ x1 x 2
_________________________________
Mixture Models
First Order Model:

q
y = ∑ βi xi
i =1
Second Order Model:

q q
y= ∑βx +∑∑β
i =1
i i
i <j
ij xi x j
Full cubic Polynomial:
q q q q
y = ∑βi xi + ∑∑βij xi x j + ∑∑δij xi x j (xi − x j ) + ∑∑∑βijk xi x j xk
i=1 i <j i <j i < j <k
Special Cubic Polynomial:
q q q
y= ∑βx +∑∑β x x +∑∑∑β
i =1
i i
i <j
ij i j
i <j <k
ijk xi x j xk
_________________________________
INTERPRETATION OF
COEFFICIENTS
• LINEAR (bi) - Average responses of pure, single

component, formulations
• QUADRATIC - (bij) - Deviation of the response

surface from a plane
y = β1x1 + β2x2 + β12x1x2

Mean
Response y=β2
0.25β12
y = β1x1 + β2x2
y=β1
x1=1, x2=0 x1=0.5, x2=0.5 x1=0, x2=1
_________________________________
The Number of Terms in the

Canonical Polynomials
# of Degree of Polynomial
factors, q Linear Quad S.Cubic F.cubic
2 2 3 - -
3 3 6 7 10
4 4 10 14 20
5 5 15 25 35
6 6 21 41 56
. . . . .
. . . . .
q q q(q+1)/2 (q2+5)/6 (q+1)(q+2)/6
_________________________________
Example: Juice Blend
• We are making a fruit punch by blending

proportions of
Orange Juice (x1)

Grapefruit Juice (x2)
Pineapple Juice (x3)
• Response of interest is
Taste of the punch (y)
Quantified on a scale of 1-25:
1 = not tasty
25 = very tasty
_________________________________
A drink is to be formulated from a

combination of orange, grapefruit and
pineapple juices; each juice may be totally
absent or present up to 100%.
1. Sketch the design space for the

drink.
2. Write down the full quadratic model.
3. How many experiments are

required?
4. Formulate a design for this test.
_________________________________
3-COMPONENT JUICE
BLEND
ORANGE GRAPEFRUIT PINEAPPLE TASTE

1 0 0 25
1 0 0 23
0 1 0 18
0 1 0 15
0 0 1 13
0 0 1 11
.5 .5 0 20
.5 0 .5 22
0 .5 .5 12
.666 .166 .167 23
.166 .666 .167 14
.166 .166 .667 15
_________________________________
_________________________________
ANALYSIS
Subset Models
Project : juice blend
Analysis : Mixture Model 1
Response Model : Taste (Y1)

Model Terms : X1, X2, X3, X1X2, X1X3, X2X3
All Possible Models Regression (using original data)
Subset Selection Algorithm : Exhaustive Search

Model
C(p) MSE Rank Model
Terms
3 10.3778 4.0944 6 X1, X2, X3
4 12.2284 4.5642 8 X1, X2, X3, X1X2
4 5.7788 2.7503 3 X1, X2, X3, X1X3
4 8.9871 3.6526 5 X1, X2, X3, X2X3
5 7.5796 3.0792 4 X1, X2, X3, X1X2, X1X3
5 10.8689 4.1364 7 X1, X2, X3, X1X2, X2X3
5 4.1626 1.9808 1 X1, X2, X3, X1X3, X2X3
6 6 2.25 2 X1, X2, X3, X1X2, X1X3, X2X3
3/18/2004 3:35:50 PM
Results
Project : juice blend
Analysis : Mixture Model 1
Response Model : Taste (Y1)
_________________________________
Model Terms : X1, X2, X3, X1X2, X1X3, X2X3



Labels
Bound Bound
X1 Orange 24.0595 0.9106 21.4722 26.6469
X2 Grapefruit 16.1071 0.9106 13.5198 18.6945
X3 Pineapple 12.0379 0.9787 9.2571 14.8187
X1X3 OrangePineapple 16.039 5.7607 -0.3285 32.4064
X2X3 GrapefruitPineapple -11.6753 5.7607 -28.0427 4.6921

Model 247.0509 3 61.7627 31.1802 0.0001 Significant
Linear Model 224.0667 2 112.0333 56.5588 4.7777E-05 Significant
X1X3 15.3551 1 15.3551 7.7519 0.0271 Significant
X2X3 8.1365 1 8.1365 4.1076 0.0823 Insignificant
Residuals 13.8658 7 1.9808
Lack of Fit 5.3658 4 1.3415 0.4735 0.7586 No LOF
Pure Error 8.5 3 2.8333
Total 260.9167 11


Response Response
Internal External
1 25 24.0595 0.9405 0.4187 0.8764 0.8599 0.1106 0.8635
2 20 20.0833 -0.0833 0.1806 -0.0654 -0.0606 0.0002 -0.0336
3 22 22.0584 -0.0584 0.7504 -0.0831 -0.077 0.0042 -0.1579
4 18 16.1071 1.8929 0.4187 1.7639 2.1911 0.4481 2.2
_________________________________
5 12 11.1537 0.8463 0.7504 1.2035 1.2512 0.8708 2.5667
6 13 12.0379 0.9621 0.4836 0.9513 0.9438 0.1695 1.0807
7 11 12.0379 -1.0379 0.4836 -1.0262 -1.0307 0.1972 -1.1802
8 15 16.1071 -1.1071 0.4187 -1.0317 -1.0373 0.1533 -1.0415
9 23 24.0595 -1.0595 0.4187 -0.9873 -0.9853 0.1404 -0.9893
10 14 15.9026 -1.9026 0.2078 -1.5188 -1.7173 0.121 -1.0406
11 23 22.1883 0.8117 0.2078 0.648 0.6187 0.022 0.3749
12 15 15.2045 -0.2045 0.2614 -0.1691 -0.1569 0.002 -0.1104
_________________________________
Model Diagnostic
Project Name : juice blend
Plot Name : Residual Plot for Taste(Y1) [Mixture Model 1]
3/18/2004 3:42:04 PM
2
-1
-2
-3
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Observed Response

Plot Name : Parity Plot for Taste(Y1) [Mixture Model 1]
3/18/2004 3:38:05 PM
26
24
Observed Response
22
20
18
16
14
12
10
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Predicted Response
_________________________________

Plot Name : Trace Plot for Taste(Y1) [Mixture Model 1]
3/18/2004 3:52:53 PM
24
Grapefruit
22
Pineapple
20
Taste
18
16
14
12
Orange
10
-0.4 -0.3 -0.2 -0.1 -0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Deviation from Reference Blend
Reference blend is the centroid of mixture.
_________________________________
Mixture Design Strategies For Response Surface Development
Three Component Mixture Design
Design Variable
Exp. X1 X2 X3
1 1 0 0
2 0 1 0
3 0 0 1
4 1/2 1/2 0
5 1/2 0 1/2
6 0 1/2 1/2
7 1/3 1/3 1/3 The center point is

8 1/3 1/3 1/3 repeated 3 times to
9 1/3 1/3 1/3 to estimate Pure Error
10 2/3 1/6 1/6

11 1/6 2/3 1/6 Lack of Fit Check Points
12 1/6 1/6 2/3
_________________________________
Multiple Constraints on the Component Proportions
In addition to general constraints:
xi ≥ 0, ∑xi =1
i =1
Lower Bound Restriction
0 ≤ Li ≤ xi , i = 1, 2 , ..., q
USE PSEUDOCOMPONENTS
_________________________________
Pseudocomponents
Create transformed xi on [0,1]
L = ∑ Li < 1
Let, i =1
x i − Li
zi =
Then, 1− L
or conversely, x i = (1 − L ) z i + Li
_________________________________
Example of Pseudocomponents
• Three ingredients are used to form a

bleach to remove glass stains
• To be effective the bleach must contain
–at least 30% bromine by weight

–at least 25% powder by weight
–at least 2% dilute HCl by weight
Thus the lower bounds are:
L 1 = 0 . 30 ≤ x 1
L 2 = 0 . 25 ≤ x 2
L 3 = 0 . 02 ≤ x 3
_________________________________
x1 = 1
x3 > 0.02
x1=0.73
x2=0.25
x3=0.02 z1 = 1
x1=0.30
x2=0.25
x3=0.45
x1 > 0.30
x1=0.30 z2 = 1 z3 = 1
x2=0.68
x3=0.02
x2 = 1 x3 = 1
x2 > 0.25
_________________________________
Design using Pseudocomponents
A {3,2} Simplex-Lattice in
Pseudocomponents
Pseudo Settings Actual Settings

bromine powder HCl bromine powder HCl
0.33333 0.33333 0.33333 0.44333 0.39333 0.16333
0 0 1 0.3 0.25 0.45
1 0 0 0.73 0.25 0.02
0 1 0 0.3 0.68 0.02
0 0.5 0.5 0.3 0.465 0.235
0.5 0 0.5 0.515 0.25 0.235
0.5 0.5 0 0.515 0.465 0.02
_________________________________
Design Strategies when Component proportions have both Upper and

Lower Bounds
• The pseudocomponents will not work

because of the upper bounds
• Use ALGORITHMIC Designs
– McLean and Anderson Algorithm

– XVERT Algorithm
– D-Optimality Algorithm
• Such a restriction is written as:
0 ≤ Li ≤ x i ≤ U i ≤ 1
Where, the Ui’s are the Upper bounds
_________________________________
Example with both Lower and Upper Bounds
Fruit Punch is made by combining juices

from watermelon, pineapple, and orange
• 40% < watermelon (x1) < 80%

• 10% < pineapple (x2) < 50%
• 10% < orange (x3) < 30%
U2 = 1.0-(L1 + L2)
= 1.0-(0.4+0.1)
= 0.5
0 . 40 ≤ x 1 ≤ 0 . 80
0 . 10 ≤ x 2 ≤ 0 . 50
0 . 10 ≤ x 3 ≤ 0 . 30
_________________________________
Design Space
Special
Quadratic Cubic
Design Design
watermelon pineapple orange watermelon pineapple orange
0.55 0.25 0.20 0.55 0.25 0.20
0.80 0.10 0.10 0.80 0.10 0.10
0.80 0.10 0.10 0.40 0.50 0.10
0.40 0.50 0.10 0.60 0.10 0.30
0.60 0.10 0.30 0.40 0.30 0.30
0.40 0.30 0.30 0.40 0.40 0.20
0.40 0.40 0.20 0.70 0.10 0.20
0.70 0.10 0.20 0.60 0.30 0.10
0.60 0.30 0.10 0.50 0.20 0.30
_________________________________
_________________________________
DESIGN WITH PROCESS

VARIABLES
• 3 Mixture variables (x1,x2,x3)

• 2 Process variables (z1,z2)
DESIGN: Mixture part run at all

combinations of process variables.
The combined design consisted of the 7
mixture blends of the {3,2} simplex
centroid lattice crossed with the four-level
combinations of the 2 process variables
2 replicates may be run at some of the 28

mixture component-process variables
combinations for LOF and error
calculations
_________________________________
Design space for mixture and

process variables
Z1, Z2 = (-1,1) Z1, Z2 = (1,1)
MIXTURE
FACTORIAL
Z1, Z2 = (-1,-1) Z1, Z2 = (1,-1)
_________________________________
Model for mixture-process variables design
Second degree model in the process variables z1, z2
y ( z ) = β0 + β1 z1 + β2 z 2 + β12 z1 z 2 + ε
Second degree model in the mixture variables x1,
x2, and x3
y ( x ) = α1 x1 + α2 x2 + α3 x3
+ α12 x1 x2 + α13 x1 x3 + α23 x2 x3 + ε
Combined model is the cross product (24 terms)
y( x, z ) = y( x ) • y( z )
_________________________________
Mixture-Process Variable Example

(Vinyl Seat)
Production of vinyl seat covers involve
several types of components, such as,
plasticizers, stabilizers, lubricants, resins,
and drying agent
An experiment was designed to study the

thickness of vinyl as a function of:
• proportions of three plasticizers (x1, x2, x3)

• levels of two process variables, extrusion rate
(z1) and drying temperature (z2)
♦ Plasticizers form 41% of the total

formulation.
♦ The proportions of the remaining

ingredients were held fixed in all blends
where the sum of their parts by weight
made up 59% of the total formulation
Project : Vinyl Seat
_________________________________
Mixture Design Task : Mixture-Process Design
Independent Variables:
Mixture Variables
Mixture Total : 1
Lower Upper
Variable Label
Bound Bound
Plasticizer
X1 0 1
1
Plasticizer
X2 0 1
2
Plasticizer
X3 0 1
3
Process Continuous Variables

Lower Upper
Variable Label
Bound Bound
X4 Extrusion Rate -1 1
X5 Drying Temperature -1 1
_________________________________
Plasticizer 1 Plasticizer 2 Plasticizer 3 Extrusion Rate Drying Temperature thickness

0.33333 0.33333 0.33333 0 0 0.9959
0 1 0 -1 -1 4.8028
0 1 0 1 1 0.2715
1 0 0 1 1 0.2791
0.5 0 0.5 1 0 1.2995
0 0 1 0 0 -0.0286
0.5 0.5 0 -1 1 -1.7562
0.5 0.5 0 -1 0 0.5371
0 0.5 0.5 -1 -1 2.2596
0 0 1 -1 1 -2.3301
0 0.5 0.5 1 -1 2.2758
1 0 0 -1 1 -3.7116
0 0.5 0.5 -1 1 0.2129
0 0 1 1 1 -2.3853
0 1 0 1 -1 4.8931
0 0.5 0.5 1 1 0.2062
0.5 0.5 0 1 -1 4.8385
0 0 1 1 -1 2.2772
0.5 0.5 0 0 0 1.5339
1 0 0 -1 -1 0.8316
0.5 0.5 0 1 1 0.2428
0.5 0 0.5 1 1 -1.0075
0.5 0 0.5 0 0 0.2966
0 0.5 0.5 1 0 1.2294
0.5 0.5 0 -1 -1 2.8941
1 0 0 1 -1 4.8976
0.5 0 0.5 -1 1 -3.0001
1 0 0 1 0 2.507
0 0.5 0.5 0 -1 2.3165
0 0 1 -1 -1 2.2491
0.5 0 0.5 -1 -1 1.5809
1 0 0 0 -1 2.892
0 1 0 -1 0 2.565
0.5 0 0.5 1 -1 3.5025
0 1 0 0 -1 4.8272
0 0 1 0 1 -2.3998
_________________________________
0 1 0 -1 1 0.2814
0 0 1 1 1 -2.3993


Labels
Bound Bound
X1 Plasticizer 1 0.5706 0.0117 0.5429 0.5983
X2 Plasticizer 2 2.5484 0.0117 2.5208 2.576
X3 Plasticizer 3 -0.0509 0.0113 -0.0775 -0.0242
X1X4 Plasticizer 1Extrusion Rate 1.9937 0.0125 1.9642 2.0232
X1X5 Plasticizer 1Drying Temperature -2.2923 0.0132 -2.3236 -2.2611
X2X3X5 Plasticizer 2Plasticizer 3Drying Temperature 5.0634 0.0727 4.8917 5.235

Linear Model 44.1097 2 22.0548 19188.0966 2.4592E-47 Significant
X1X4 29.2733 1 29.2733 25468.3316 1.6563E-45 Significant
X1X5 34.4812 1 34.4812 29999.3061 1.4243E-46 Significant
X2X5 30.0932 1 30.0932 26181.6457 1.095E-45 Significant
X3X5 34.4652 1 34.4652 29985.3742 1.4343E-46 Significant
X2X3X5 5.5705 1 5.5705 4846.4714 9.914E-35 Significant
Residuals 0.0345 30 0.0011
Lack of Fit 0.0344 29 0.0012 12.0985 0.2242 No LOF
Pure Error 9.8E-05 1 9.8E-05
Total 201.7165 37

_________________________________

Response Response
Internal External
1 0.9959 1.0227 -0.0269 0.0272 -0.8033 -0.7984 0.0023 -0.2587
2 4.8028 4.8405 -0.0377 0.2519 -1.2854 -1.3001 0.0695 -1.4608
3 0.2715 0.2563 0.0152 0.3358 0.5484 0.5419 0.019 0.7461
4 0.2791 0.272 0.0071 0.4059 0.2721 0.2679 0.0063 0.4287
5 1.2995 1.2567 0.0428 0.0745 1.3111 1.3276 0.0173 0.7293
6 -0.0286 -0.0509 0.0223 0.111 0.6961 0.69 0.0076 0.4721
7 -1.7562 -1.7296 -0.0266 0.1703 -0.8626 -0.8588 0.0191 -0.7534
8 0.5371 0.5626 -0.0255 0.0884 -0.7879 -0.7828 0.0075 -0.4719
9 2.2596 2.2856 -0.026 0.2315 -0.8737 -0.8702 0.0287 -0.9249
10 -2.3301 -2.364 0.0339 0.202 1.1206 1.1256 0.0397 1.0968
11 2.2758 2.2856 -0.0098 0.2315 -0.3287 -0.3237 0.0041 -0.3441
12 -3.7116 -3.7154 0.0038 0.4809 0.1572 0.1547 0.0029 0.2883
13 0.2129 0.212 0.0009 0.2709 0.0324 0.0319 4.8763E-05 0.0376
14 -2.3853 -2.364 -0.0213 0.202 -0.7021 -0.6961 0.0156 -0.6782
15 4.8931 4.8405 0.0526 0.2519 1.7939 1.8668 0.1354 2.0975
16 0.2062 0.212 -0.0057 0.2709 -0.1983 -0.1951 0.0018 -0.2303
17 4.8385 4.8486 -0.0101 0.1355 -0.3197 -0.3148 0.002 -0.2414
18 2.2772 2.2623 0.0149 0.3305 0.5362 0.5298 0.0177 0.7207
19 1.5339 1.5595 -0.0256 0.0499 -0.7749 -0.7696 0.0039 -0.3417
20 0.8316 0.8692 -0.0376 0.3994 -1.4318 -1.4584 0.1704 -2.3031
21 0.2428 0.2642 -0.0213 0.1603 -0.6864 -0.6802 0.0112 -0.5755
22 -1.0075 -1.046 0.0385 0.1345 1.2211 1.2315 0.029 0.9401
23 0.2966 0.2599 0.0367 0.0496 1.1114 1.116 0.0081 0.4938
24 1.2294 1.2488 -0.0194 0.0492 -0.5861 -0.5796 0.0022 -0.2554
25 2.8941 2.8549 0.0392 0.1434 1.2508 1.2632 0.0327 1.0009
26 4.8976 4.8567 0.0409 0.3457 1.493 1.5257 0.1472 2.1477
27 -3.0001 -3.0397 0.0396 0.1544 1.2714 1.2852 0.0369 1.0633
28 2.507 2.5643 -0.0573 0.2234 -1.919 -2.0144 0.1324 -2.0923
29 2.3165 2.2856 0.0309 0.2315 1.0408 1.0423 0.0408 1.1078
30 2.2491 2.2623 -0.0132 0.3305 -0.4767 -0.4705 0.014 -0.6401
31 1.5809 1.5658 0.0151 0.1622 0.4877 0.4814 0.0058 0.4103
32 2.892 2.8629 0.0291 0.2368 0.9814 0.9808 0.0373 1.0578
33 2.565 2.5484 0.0166 0.1192 0.5215 0.5151 0.0046 0.367
_________________________________
34 3.5025 3.5595 -0.057 0.1457 -1.8188 -1.8958 0.0705 -1.5163
35 4.8272 4.8405 -0.0133 0.2519 -0.4534 -0.4473 0.0086 -0.5026
36 -2.3998 -2.364 -0.0358 0.202 -1.1809 -1.189 0.0441 -1.1586
37 0.2814 0.2563 0.025 0.3358 0.9059 0.9032 0.0519 1.2435
38 -2.3993 -2.364 -0.0353 0.202 -1.1644 -1.1716 0.0429 -1.1416
Project Name : Vinyl Seat(C:\Program Files\Stat-Studio\samples\Design\Vinly Seat.ssp)

Plot Name : Parity Plot for thickness(Y1) [Mixture Model 2]
3/24/2004 9:16:23 PM
3
Observed Response
-1
-2
-3
-4
-4 -3 -2 -1 0 1 2 3 4 5
Predicted Response
_________________________________
Project Name : Vinyl Seat(C:\Program Files\Stat-Studio\samples\Design\Vinly Seat.ssp)

Plot Name : Variable Importance Plot for thickness(Y1) [Mixture Model 2]
3/24/2004 9:16:36 PM
Plasticizer 3*Drying Temperature -0.4661

Model Terms
Plasticizer 2 0.4055
Plasticizer 1*Extrusion Rate 0.3838
Plasticizer 2*Plasticizer 3*Drying Temperature 0.1988
Plasticizer 1 0.0908
Plasticizer 3 -0.0083
-0.5 -0.4 -0.3 -0.2 -0.1 -0.0 0.1 0.2 0.3 0.4
Model Terms : Plasticizer 1*Drying Temperature

Standardized Estimates : -0.4262
_________________________________
_________________________________
Screening Design
• In chemical industries there is often a

large number of potentially important
ingredients for consideration (q > 5)
• Reduction in q is sought through

screening experiments to facilitate
◊ understanding of the system

◊ reduction in cost
In screening experiments, q components

are fit to the data using a linear model
y$ ( X ) = b1 x1 + b2 x 2 +...+ bq x q
_________________________________
Screening Design (contd.)
Component Effect: The effect of

component i on the response is the
change in the response when xi is
changed keeping the relative
proportions of the other components
constant.
Such a change can be made on the

rays
x1 = 1
RAYS
x1 = 2x3
x2 = 1 x3 = 1
x2 = x3
_________________________________
EXAMPLE:
Project : Octane-Blending Experiment
Mixture Design Task : Mixture Screening(4)
Mixture Variables
Mixture Total : 1
Lower Upper
Variable Label
Bound Bound
X1 Straightrun 0 0.21
X2 Reformate 0 0.62
X3 Naptha 0 0.12
X4 Cat. Naptha 0 0.62
X5 Polymer 0 0.12
X6 Alkylate 0 0.74
X7 Nat Gasoline 0 0.08
Design Properties:
Mixture Design Strategy : Mixture-Screening
Number of points needed to estimate the model : 7
Overall Centroid Points : 1
Number of repeats : 1
Total number of points : 9
_________________________________
Mixture Candidate Points :
Number of
# Type Used
points
1 Vertices + 112
2 Axial check blends 112
3 Constraint plane centroids 14
Total number 112
DATA SHEET
Reformat Cat. Nat Motor
Straightrun e Naptha Naptha Polymer Alkylate Gasoline Octane
0.20999999 0 0.12 0.62 0 0.05 0

0.20999999 0.62 0.05 0 0.12 0 0
0 0 0 0.62 0.12 0.26 0
0 0.14 0.12 0 0 0.74 0
0.20999999 0 0 0 0 0.71 0.08
0 0.3 0 0.62 0 0 0.08
0.20999999 0 0.12 0.47 0.12 0 0.08
0.10535714 0.234911 0.059196 0.234911 0.059196 0.266875 0.039554
0 0 0.12 0.62 0 0.18 0.08
_________________________________
DATA SHEET (including Constraint Plane Centroids)

Straightru Cat. Nat Motor
n Reformate Naptha Naptha Polymer Alkylate Gasoline Octane
0 0.62 0 0.38 0 0 0
0.0500000
0.20999999 0 0.12 0.62 2 0 0
0.6700000
0.20999999 0 0 0 0.12 2 0
0.0900000
0.20999999 0.62 0 2 0 0 0.08
0.0599999
0 0.62 0.12 0 0.12 9 0.08
0.1799999
0 0 0 0.62 0.12 9 0.08
0.7400000
0 0 0.12 0.06 0 1 0.08
0.2508000 0.2508000 0.3355999
0 1 0.0612 1 0.0612 9 0.0404
0.0624074 0.3101851 0.0624074 0.4046296 0.0416666
0.1187037 0 1 9 1 5 7
0.2453703 0.2453703 0.0620370 0.2950000
0.11092592 7 0 7 4 2 0.0412963
0.3101851 0.0624074 0.0624074 0.4046296 0.0416666
0.1187037 9 1 0 1 5 7
0.2453703 0.0620370 0.2453703 0.2950000
0.11092592 7 4 7 0 2 0.0412963
0.3453846 0.0653846 0.3453846 0.0653846 0.0438461
0.13461538 3 1 3 1 0 5
0.2409259 0.0605555 0.2409259 0.0605555 0.2890740
0.10796296 2 5 2 5 6 0
0.2352083 0.2352083 0.1806250 0.0352083
0.20999999 3 0.051875 3 0.051875 1 3
0.0570588 0.0623529 0.0570588 0.0623529
0.10294117 0.62 2 4 2 4 0.0382353
0.2273076 0.2273076 0.0561538 0.2400000 0.0376923
0.09153846 9 0.12 9 5 1 1
0.0623529 0.0570588 0.0570588 0.0623529
0.10294117 4 2 0.62 2 4 0.0382353
0.2273076 0.0561538 0.2273076 0.2400000 0.0376923
0.09153846 9 5 9 0.12 1 1
0.0510714 0.0510714 0.7400000 0.0339285
0.05892857 0.0325 3 0.0325 3 1 7
0.2319230 0.0576923 0.2319230 0.0576923 0.2461538
0.09461538 7 1 7 1 5 0.08
0.2349107 0.0591964 0.2349107 0.0591964 0.0395535
0.10535714 1 3 1 3 0.266875 7
0.7400000
0 0 0.12 0.14 0 1 0
_________________________________
Alternate Model Forms
INCLUSION OF INVERSE TERMS (1/xi):
To account for rather large changes in

response when a factor is close to 0 or 1
◊ may result in better fit than with high degree

polynomials
e.g., as proportion of oil in the fuel mixture of a two-

cycle internal combustion engine tends to zero, the
engine performance becomes erratic
Models:
q q
y = ∑ β i x i + ∑ β − i x i−1
i =1 i =1
q q q
y = ∑ β i x i + ∑ ∑ β ij x i x j + ∑ β − i x i− 1
i =1 i <j i =1
_________________________________
Project : Gasoline Octane

Mixture Design Task : Mixture Design with inverse term
Mixture Variables
Mixture Total : 1
Lower Upper
Variable Label
Bound Bound
X1 X1 0.01 0.648
X2 X2 0 0.954
X3 X3 0.036 0.924
Responses Models:
Variable Label Mixture Model

Y1 Octane Number X1, X2, X3, X1X2, X1X3, X2X3, 1/X1, 1/X2, 1/X3
_________________________________
X1 X2 X3 Octane Number
0.278 0.267 0.454 83.45593658
0.648 0 0.352 89.35029034
0.01 0.954 0.036 126.8244268
0.648 0.316 0.036 99.16556145
0.076 0 0.924 60.35789469
0.01 0.066 0.924 86.21433551
0.01 0.51 0.48 110.2195226
0.648 0.032 0.32 90.99558018
0.074 0.801 0.125 102.1432641
0.52 0.355 0.125 99.97015188
_________________________________
Project Name : Gasoline Octane

Plot Name : Variable Importance Plot for Octane Number(Y1) [Mixture Model 1]
3/24/2004 10:22:47 PM
X2 1.9799
X1 1.794
Model Terms
X3 0.9299
1/X1 0.809
1/X3 -0.1682
-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
_________________________________
USE OF RATIOS:
In some systems, component ratios

determine properties of a blend
◊ For q variables, (q-1) ratios will define the
system
e.g., in the manufacture of some porcelain

glasses, the ratio of silica to soda (w1) and the
ratio of silica to lime (w2) is a more meaningful
way to study these three ingredients than to
look at the actual proportions of each in a
blend.
x1 x2
w1 = , w2 = , x1 + x 2 + x3 = 1
x2 x3
_________________________________
Model Reduction
• Delete insignificant terms: high levels of

significance or low t-ratios for higher order
terms, in a model with no lack of fit.
• Combine terms: two terms having

coefficients of nearly equal magnitude and
insignificant interaction have similar blend
behaviors and could be combined.
• Search for model with minimum mean

square error .
• Check model order by evaluating the

contribution of the combined set of quadratic
(etc.) terms
_________________________________
Lack of Fit: Linear Model
Observed And Predicted

(Second Order Model Indicated)
80LIFE
70
60
^ _ _ ^
50
(y - y) = (y - y) + (y - y)
tot err pe lof
40 TEMP
0 10 20 30 40 50 60 70 80 90 100
_________________________________
R-SQUARE
INTERPRETATION
• R-Square (R2) calculated from a no

intercept model is not meaningful since
the sum of squares corrected for the
mean is not removed
• A more reasonable R-Square is the

square of r, the simple correlation
between the observed y's and the
predicted values
_________________________________
Multi-Component Mixture Constraints
• In addition to upper and lower bounds,

there may exist m linear constraints
among the components:
C j ≤ β1 j x1 + β2 j x 2 +...+ βqj x q ≤ D j , j = 1,2,..., m

where βij are scalar constants
• Multi-component constraints can cause

major changes in the experimental region
• Stat-Studio can handle that

(in realm of computer-aided design)
_________________________________
REFERENCES
John A. Cornell - Experiments with Mixtures:

Designs, Models and the Analysis of Mixture
Data. 2nd ed. New York: Wiley and Sons, 1990.
John A. Cornell - How To Run Mixture

Experiments for Product Quality. (The ASQC
References in Quality Control, Vol 5).
ISBN: 0-87389-021-3
I. N. Vuchkov - Catalogue of Sequentially

Generated Designs (Bulgarian Institute of
Chem. Tech.)
_________________________________

Class Notes PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Class Notes PDF

Загружено:

Авторское право:

Доступные форматы

EXPERIMENTAL DESIGN STRATEGIES

A Short Course for Chemical Engineering Graduate Students

Debashis Neogi, Ph.D. - Lead Systems Engineer

Computational Modeling Center

SECTION II: Regression Analysis

SECTION III: Factorial and Screening Design Strategies

SECTION IV: Response Surface Concepts

SECTION V: Mechanistic Model Building

SECTION VI: Mixture Design

About This Course...

The primary focus of the seminar is to develop an understanding of the basics of

The instructors will appreciate suggestions on how to improve the course.

1. Barker, Thomas B. (1985), Quality By Experimental Design, Marcel Dekker Inc.

8. Myers, Raymond H. and Montgomery, D. C. (1995), Response Surface Methodology, Virginia

Statistical methods are aids to engineering judgment and not a

Always graph your data in some simple way - always.!!

WHAT IS DESIGN OF EXPERIMENTS? ...................................................................................................... 3

A TYPICAL STRATEGY ...................................................................................................................................... 4

ABOUT THIS COURSE... .................................................................................................................................... 5

WHAT IS DESIGN OF EXPERIMENTS?

Main objective of most experiments -- assess the

e.g., estimate the effects of temperature and pressure on

Typically, we assume a relationship between

The function φ represents an approximation to the true

Mechanistic models based on first principles may be

The challenge is to find this relationship with a

Objectives Experimental Design

Data Collection Raw Data

ABOUT THIS COURSE...

This course will emphasize four major topics:

• Experimental design strategies for empirical

• Response Surface Methodology

• Mechanistic Model Building

A one-at-a-time approach: vary one factor while holding

Let's see how good the strategy is...

We need measures of performance to evaluate the

Two such measures are:

Measure Meaning Criterion

R2 - Coefficient of A measure of the 0 ≤ R 2 ≤ 1,

RMSE - Mean Unbiased estimate of σ2 The smaller the better

On the surface, this appears to be a good model...

The model is:

(Note, however, that when we tried the full quadratic

The model performs well for the chosen points...

Dep Var Predict Std Err Lower95% Upper95%

1 38.9900 38.8207 0.686 36.5045 41.1370 0.1693

Let's see how well it predicts...

Very poor data structure!!!

• Approach didn't permit estimation of the true model

• Made an incomplete and incorrect model appear to be

• Regression model produced biased estimates (at best!);

In general, if you use a one-at-a-time approach be

• Partial confounding among the predictors - won't be

• Time and energy wasted in dealing with data errors,

• We could have used a factorial design which requires

• Let's assume we have the luxury of 15 observations...

The best (parsimonious) model...

Response Model : Y1 (Y1)

Number of terms in the User Selected Full Model : 10

Final/Best Subset Model

95% Confidence Interval

X1 X1 2.6063 0.5681 1.1099 4.1026

X2 X2 1.7075 0.5681 0.2112 3.2038

y =11.8434 + 2.6063X1 + 1.7075X2 - 1.19X3 + 8.0844X1*X2