DOE Design & Analysis

Using Minitab

L. Goch February 2011

AGENDA
DOE Design
DOE Pitfalls & Types of Designs
Screen Design Example
Characterization Design Example
DOE Des ign &
Optimization Design Example Analys is us ing Minitab.mtb

DOE Analysis
Response Surface Design
EXPERIMENTS PITFALLS
Having an unknown or unaccounted for input
variable be the real reason your Y changed
These are called Noise Variables
Number of storks correlating to human births
Solution: Randomization
Having too little data in too short a time period
Murphy at work again.
Solution: Repetitions within Each Run
Studying a local event and believing it applies to
everything
Same as sample size selection.
Solution:Replication of Runs within the DOE or as
a Confirmation DOE
HIGH LEVEL MAP OF EXPERIMENTS

Screening Designs Plackett-Burman DOE

(6-11 Factors) L16 & L18 DOEs

Characterization Fractional Factorial &

Designs (3-5 Factors) Full Factorial DOEs

Optimization Response Surface DOEs

Designs (<3 Factors)
 SCREENING DESIGNS
PLACKETT-BURMAN EXAMPLE (2 LEVEL DOE)
STAT > DOE > FACTORIAL > CREATE FACTORIAL
DESIGN
CHECK PLACKETT-BURMAN DESIGN
WILL REVIEW DURING TRAINING

L16 & L18 ARE ALSO GOOD SCREENING

DESIGNS (2 & 3 LEVEL MIXED DOE)
STAT > DOE > TAGUCHI > CREATE TAGUCHI
DESIGN
CHECK MIXED LEVEL DESIGN
REVIEW ON OWN
LETS USE MINITAB TO GENERATE THE
MATRIX 1. Choose design 2. Choose factors

3. Choose the
final design

NOTE: MINITAB will

always default to the Exp.
with the fewest runs
 4. Define the Factors and
DESIGN MATRIX their levels

5. Hit OK after you have named all

your factors and their levels. Levels
can be alphanumeric unless when
center points are used.

Enter Factors most likely to have

Interactions FIRST!
DESIGN MATRIX OUTPUT
STANDARD ORDER SCREENING
EXPERIMENT

Minitabs default is to display

the runs in Random Order.
 CHARACTERIZATION
DESIGNS
FULL FACTORIAL DOE
STAT > DOE > FACTORIAL > CREATE FACTORIAL
DESIGN
CHECK GENERAL FULL FACTORIAL DESIGN
REVIEW ON OWN

FRACTIONAL FACTORIAL DOE

STAT > DOE > FACTORIAL > CREATE FACTORIAL
DESIGN
CHECK 2-LEVEL FACTORIAL (DEFAULT
GENERATORS)
WILL REVIEW DURING TRAINING
DOE EXAMPLE
Problem: Current Car gas mileage is 30 mpg.
Would like to get 40 mpg.
We might try:
Change brand of gas
Change octane rating
Drive Slower
Tune-up Car
Wash and wax car
Buy new tires
Change Tire Pressure
What if it works?
What if it doesnt?

Survey Says These variable greatly effect MPG

LETS USE MINITAB TO GENERATE THE
MATRIX 1. Choose design type 2. Choose # factors

WHAT
DESIGN
SHOULD
YOU
CHOOSE?
LETS USE MINITAB TO GENERATE THE
MATRIX 1. Choose design 2. Choose factors

3. Choose the final

design

WHAT
DESIGN
SHOULD
YOU
CHOOSE?
 DESIGN MATRIX 4. Define the Factors and
their levels

5. Hit OK after you have named all

your factors and their levels. Levels
can be alphanumeric except when
centerpoints are used.
 DESIGN MATRIX 6. Click on the Options button
so we can de-select something
for this exercise...

7. Turn off the Randomization

option for this exercise only
DESIGN MATRIX OUTPUT
STANDARD ORDER FOR FULL
FACTORIAL
 OPTIMIZATION DESIGNS
BOX BEHNKEN & CENTRAL COMPOSITE
DESIGNS
STAT > DOE > RESPONSE SURFACE > CREATE
RESPONSE SURFACE DESIGN
CHECK BOX BEHNKEN OR CENTRAL COMPOSITE
LETS USE MINITAB TO GENERATE THE
MATRIX 1. Choose design 2. Choose factors

3. Choose the final

design

WHAT
DESIGN
SHOULD
YOU
CHOOSE?
 DESIGN MATRIX
4. Define the Factors and
their levels

5. Hit OK after you have named all

your factors and their levels.
Factors MUST be numeric.
Choose Cube or Axial Points
DESIGN MATRIX OUTPUT
RANDOM ORDER FOR CENTRAL COMPOSITE
DESIGN

Axial Points
are the Actual
Max & Min
Points of the
Design.
 ANALYZING DATA
FULL & FRACTIONAL FACTORIAL DOE
STAT > DOE > FACTORIAL > DEFINE CUSTOM
FACTORIAL DESIGN
ANALYZE FACTORIAL DESIGN
REVIEW ON OWN

RESPONSE SURFACE DOE

STAT > DOE > RESPONSE SURFACE > DEFINE
CUSTOM RESPONSE SURFACE DESIGN
ANALYZE RESPONSE SURFACE DESIGN
REVIEW ON OWN
MINITAB PROCEDURES: DATA ANALYSIS
WITH MULTIPLE INPUTS (XS) AND ONE
OUTPUT (Y)
We can use the Analyze Response Surface Design
feature under DOE to analyze any type of data
collection with multiple inputs (Xs)
Used for 2k Full & 2k-n Fractional Factorials or other
Characterization or Optimization designs
Used for Plackett-Burman or other screening designs
Used for Passively Collected data
Used for Historically Collected data
Can NOT be used when an Input is Non-Numeric and
has more than 3 levels (e.g. 3+ Machines, 3+ Cavities)

Remember CAUSATION can only be determined thru

experimentally designed and collected data
ROADMAP FOR ANALYZING MULTIPLE
INPUTS (XS):
Step 1: Identify inputs (Xs) vs outputs (Ys).

Step 2: Plot your data

Step 3: Find Best Equation based on P-Values

Step 4: Check R-squared and Adj. R-squared

Step 5: Determine how well your model (i.e. equation) can

predict.

Step 6: Check Residuals

Step 7: Make 3-D plots

Step 8: Do the Results Make Sense?

Step 9: Confirm Results or begin next Experiment
ANALYZE THE DATA Open worksheet
Carpet.mtw

Step 1) Identify
Inputs & Outputs

Inputs: Carpet
Composition
Output: Durability

Step 1b) Composition can be

coded from text to numeric since
it has only 2-levels.
Carpet Type can NOT be coded
since its non-numeric & 4-
levels.
ANALYZE THE DATA Open worksheet
Reheat.mtw

Step 1) Identify
Inputs & Outputs

Inputs: Operator
Temp
Time
Output: Durability

Step 1b) Operator can be

coded from text to numeric
since it has only 2-levels.
ANALYZE THE DATA
Step 2) Plot the data

3D Scatterplot of Quality vs Time vs Temp

Operator
A
B

7.5

Quality 5.0

2.5 40
35
0.0
30 Time
350
400 25
450
Temp

Does there appear to be any patterns in the data?

ANALYZE THE DATA Step 3) Find Best Equation
Based on P-values
* Define Inputs in MINITAB

Select
Inputs

Click OK
ANALYZE THE DATA Step 3) Find Best Equation
Based on P-values
* Define Inputs in MINITAB

Inputs Defined in MINITAB

 Step 3) Find Best Equation
ANALYZE THE DATA Based on P-values
* Analyze Data

Select Select
Terms & Output
Click OK
ANALYSIS Step 3) continued

MINITAB tells you there is not enough

information to get p-value on these terms.

P-Values!
 FINDING THE BEST MODEL Step 3) continued

Remove term from Equation Terms

One at a time
remove highest
P-value >0.10
until all <0.10

Now we can reduce the model more by removing the 2 input terms
that are significantly above our alpha value of 0.10
 TERM Step 3) continued

Press
ELIMINATION
<Ctrl> e

Click
Terms

Double Click
on Terms to
Eliminate
 FINDING THE BEST MODEL Step 3) continued

One at a time
remove any
two input
terms with
p>0.10

Continue reducing the model by removing the 2 item terms that are
significantly above our alpha value of 0.10
FINDING THE BEST MODEL
Step 3) continued

One at a time remove

any main effect terms
with p>0.10 if they are
NOT in a 2 input term.

Continue reducing the model by removing the main effect terms that
are significantly above our alpha value of 0.10
 FINDING THE BEST MODEL
Step 3) continued

Evaluate any terms

with p>0.05 if they are
NOT in a 2 input term.

Evaluate any term with an alpha value of >0.05. These are marginally
significant terms. Only leave in if 1) that are contained in a significant
2 input term OR 2) they make sense per theory/prior testing.
FIND THE BEST MODEL Step 3) completed

This is our best equation to describe our Quality level

based on the p-values

All Terms in the Regression Equation are Significant

The p-values are < 0.05.
FIND THE BEST MODEL Step 3) completed

Frozen Food Quality = -180.963 + (0.43070 * Temp) +

(5.79598 * Time) - (0.000318 * Temp2) - (0.05181 * Time2) -
(0.00521 * Temp * Time)
ANALYZE THE R-SQUARED(S)
Step 4) Check R-squared
and Adj. R-squared

If more
than ~4%
apart
eliminate
term with
highest p-
value

Temp & Time explain 71.5% of the variability in Quality

HOW ACCURATE IS THE MODEL?
Step 5) Determine Model
Accuracy

Equation
can
predict to
within +/- 2
Stdevs

Model can Predict Quality to within +/- 3.4 with a

95% Confidence Level
 ANALYZE THE RESIDUALS
Step 6) Check Residuals
Press
<Ctrl> e

Click
Graphs

Check Four in One

ANALYZE THE RESIDUALS Step 6) Check Residuals

Looking for
Normal Looking for
Distribution Random Pattern
Residual Plots for Quality
Normal Probability Plot Versus Fits
99
4

90
2

Residual
Percent

50 0

-2
10

1 -4
-4 -2 0 2 4 -5 0 5
Residual Fitted Value

Histogram Versus Order

4
10.0
2
7.5
Frequency

Residual

0
5.0

-2
2.5

0.0 -4
-3.2 -1.6 0.0 1.6 3.2 1 5 10 15 20 25 30 35 40 45
Residual Observation Order

Residual Plots: Use if n > 25

PLOT THE RESULTS
Step 7a) Make 3-D
Plots

Select

Check Surface Plot

& Click Setup
PLOT THE RESULTS Step 7a) Make 3-D Plots

Surface Plot of Quality vs Time, Temp

Quality
0
39
36
33
-5
30 Time
350 27
400
450 24
Temp

Best Quality at Low Temp & High Time.

Robust at ~350-425o & ~33-38 minutes.
 EVALUATE THE Step 8) Does the Results
RESULTS Make Sense

EXPERIMENTAL RESULTS:
Numbers results matched up with original plotted data.
Operator didnt matter to the results.
Lower oven temps & longer times result in the highest,
most robust quality levels.

Are the results what you would have expected?

Are some statistically significant items not PRACTICALLY
significant?
Looking at the 3-D plot, do the changes in Temp & Time
have a big enough effect on Quality to be useful?
 CONFIRM RESULTS!
Step 9) Confirm Results or begin Next Experiment

ALWAYS, ALWAYS run a confirmation run at the optimal

settings or a small confirmation experiment. This is critical
to ensure that your results are accurate!!!!
If your data was historical or collected passively, you will
need to run an experiment to show that your inputs
CAUSED the changes to happen in your output.
At this point you may decide to eliminate factors from your
experimentation process or add new factors to your
experimentation.
Be careful to set up your next experiment so that the
results can be compared to your previous experiment(s).
CONFIRM RESULTS!
Step 9) Confirm Results
* Determine Optimal Settings
Step 9) Confirm Results (cont.)
* Determine Optimal Settings

Select
Output
Variable
Enter
Specifications
 Step 7b) Make
PLOT THE RESULTS Optimization Plot
Optimal Temp Time
D High
Cur
475.0
[350.0]
38.0
[38.0]
0.00000Low 350.0 24.0

Quality
Maximum
y = 6.8832
d = 0.00000

Click & Drag Red lines to see changes in Output & Relationships
Run confirmation at 350o for 38 minutes for maximum Quality.
SUMMARY
The goal of DOE design is to get the most
information from the fewest amount of runs.
Thus, DOE design is based on specific
combinations of
1) the # of Factors to be tested
2) the # of Levels for each of the factors

The goal of DOE analysis is to achieve reliable,

predictable results. For this to happen, four
items must be evaluated as part of the analysis
1) P-values: Significance of Terms in Equation
2) R-Square: Relationship of Inputs to Outputs
3) +/- 2 * S: Predictability of Equation
4) Residuals: Violation of Analysis Assumptions