I1 - Experiment Design Tools

Six Sigma
Experiment Design Tools
Experiment Design Tools
Most companies collect data to monitor the health of their processes, but most of this data is not acted on in any
way because it does not contain meaningful information. Often, when these compa-nies decide to perform an
experiment in an effort to produce more meaningful data on a specific CT characteristic, they design a one-
factor-at-a-time test, but progress is slow and expensive using this technique. On the other hand, companies
who learn to use Design of Experiments (DOE) are able to leverage their efforts by conducting well thought out
experiments where several factors are altered simultaneously in one efficient series of tests. DOE provides a
wealth of information and in a shorter period of time.
The role of a statistically designed experiment is to identify the most influential factors (vital few) associated with
a particular CT characteristic and to define their relationships using analytical quantities. Since interactions
between various factors are also defined, a single DOE can yield many revealing facts allowing the
experimenters to quickly improve their process.
It is most effective to conduct DOE's early in the design process, experimenting with the effect of tolerancing and
failure modes, so that when the product or service is introduced to the market place, customer satisfaction will be
high and warranty claims minimal. The same experimental techniques can then be used throughout the life cycle
to continuously improve, taking into account new information as it becomes available.
There are two components to a DOE: Experimental Design and Analytical Procedure. It is best to think these out
carefully before conducting the DOE in order to produce the best result. Depending on the level of understanding
available about a process, there are various types of DOE that can be employed. Screening designs are typically
used early in the process where many factors are involved, Characterization designs narrow the number of
factors down to only a few, and Optimization designs focus on only one or two factors, but in much more depth.
To ensure that a DOE is designed correctly, it is necessary to consider such topics as Yates standard order,
factor levels using the –1, 1 notation, column contrast, balance, orthogonality, confounding and interactions
between factors. Also, there are design factors that influence the time and cost aspects of the DOE, such as
replications and the basic design type.
The two-level two-factor design is relatively common in practice and makes a good starting point for most
problems. When replications are taken into account, very meaningful results can be achieved with as few as 8 to
16 runs. When you don't know which factor has a substantial effect on a response variable, and you want to
verify several factors, then the fractional factorial design is the preferred analytical tool. It is common practice to
do a fractional factorial experiment when we have five factors or more. In this case, you only conduct runs which
represent a portion of the matrix, because, for example, a 6-factor two-level experiment with no replications
would involve 64 runs. The advantage, therefore, of a fractional factorial experiment is that fewer tests are
required.
When it comes to analysis of the data from a DOE, we must first evaluate the statistical significance. This is
done by computing the One-Way ANOVA or the N-Way ANOVA (for more than one factor). The practical
significance can be evaluated through the study of sums of squares, pie charts, Pareto diagrams, main effects
plots, and normal probability plots. Note that there are situations where factors are statistically significant, but not
practically significant. In any analysis, it is important to analyze the residuals prior to any conclusions being
drawn. A step-by-step approach to designing and conducting any DOE must be adopted. The use of cross-
functional teams in the selection of factors to study, the design of the experiment, conducting the experiment,
analyses of the data, recommendations, and implementation of DOE results all require careful planning and
attention to detail to achieve success.
Key Questions
• What is a "statistically designed experiment" and what roles do they play in Six Sigma?
• What are the guiding principles surrounding statistically designed experiments?
• How can the practical and statistical influence of a single CTP be exposed?
• How can the practical and statistical influences of two or more CTP's be exposed?
• How can conflicts between competing performance objectives be resolved?
• What is the process and guidelines for planning statistically designed experiments?
Key Questions
A DOE is a systematic method where a number of factors are changed simultaneously, following a
predetermined pattern to investigate their effect on the response. The role of a statistically designed experiment
is to determine the most influential factors associated with a particular CT characteristic (vital few) and to
understand their relationships using analytical quantities.
The fundamental guiding principles surrounding statistically designed experiments are: Process Relationships,
Replication, Randomization, and Experimental Control.
The statistical influence of a single CTP can be exposed through hypothesis testing using tools such as One-
Way ANOVA, and the practical influence can be exposed through the study of sums of squares presenting the
results in pie charts or Pareto diagrams.
The statistical influence of two or more CTP's can be exposed through hypothesis testing using tools such as N-
Way ANOVA, and the practical significance can be exposed with the use of graphs such as main effects and
normal probability plots.
Conflicts between competing performance objectives can be resolved by the use of interaction effects between
factors in order to find the best compromise settings.
The process and guidelines for planning statistically designed experiments are: Define the problem, set the
objective, select response variables, select factor levels, conduct experiment(s), analyze results, and ensure
results are well understood and communicated.
Why/When to Experiment
TO IDENTIFY THE VITAL FEW DO AS EARLY AS POSSIBLE IN

TO ENSURE A ROBUST DESIGN PRODUCT LIFE CYCLE
TO SAVE MONEY TO MAXIMIZE DESIGN QUALITY
High
Focus on defect detection

and correction
Y = f(X1, X2, …, Xn)
 
Cost to deal
with poor CTQs CTPs
quality or
CTCs 
Focus on defect or
prevention CTDs Factors
Improve
Low
1_05_01_002
Product Process Manufacturing Field

Design Design
Experiment Design Tools 3
Why/When to Experiment
At a planning session, Six Sigma team members are evaluating whether or not they should conduct an
experiment on their process. They ask the Black Belt to outline a criteria for this. The Black Belt explains why
and when to use experiments:
• Experimentation can be used to improve the performance, quality and cost associated with key customer
requirements.
• We use them to confirm and quantify the important inputs (CTPs) to our processes.
• Determine the "vital few" variables when there are no analytical equations to guide us.
• Experimentation can significantly reduce new product (or service) development cycle times and it is
conceivable that manufacturing tolerances can be relaxed.
• A robust design can be created if we test it prior to design freeze. A product or service that is robust to
variations in input factors and operating environment will cost society less over the life of the product,
resulting in its customer loyalty.
• The need for product inspection and performance testing can be significantly reduced with extensive
experimentation.
• We should always experiment as early in the cycle as possible, so as to improve quality, thus reducing
rework, warranty claims and dissatisfied customers.
The Black Belt draws the above chart on the blackboard, summarizing his point that we should begin to test key
business opportunities with a series of well planned experiments as early in the development cycle as possible in
order to reduce the cost of poor quality, and in addition, throughout the product life cycle to achieve continuous
improvement.
Fundamental Principles of Experimentation
1. Understand Process Relationships
• Among process variables
• Among response variables
2. Replication
• Increased precision of effect estimates
• Provide measure of experimental error
3. Randomization
• Select experimental units randomly
• Randomize run sequence
4. Experimental Control
• Active variation of process variables
• Control other sources of variation
Fundamental Principles of Experimentation
During Black Belt training, a new Black Belt asks the Instructor to review the fundamentals of Experimentation.
The Instructor outlines the four basic principles, using the example of the classical apple falling on the head of
Isaac Newton.
"The first, and probably the most important principle", says the Instructor, "is to fully understand the process. In
this example, the energy imparted by the falling apple is the process of interest to Isaac Newton. He set out to
design a series of experiments to determine the relationships between the mass of the apple and the fall
distance as they relate to the energy imparted."
"Replication is the second principle of the experimentation process. Usually, two to five replications are
conducted for each set of conditions in order to ensure that the "white noise" is captured. Replication allows a
measure of experimental error."
"The third characteristic of an experiment is to ensure that the order of conducting a series of runs is randomized
in order to avoid false conclusions due to factors which may vary over the course of the experiment, such as
temperature, wind velocity, etc. These variables will probably have small effects on the results, which will result
in experimental error."
"The fourth fundamental principle of experimentation is what is known as experimental control. This simply
means that you deliberately change some variables, while carefully controlling all other variables as much as
possible, noting the effect."
Role of Experimentation
Experimentation - the manipulation of controllable factors ( indepen -

dent variables ) at different levels to see their effect
on some response (dependent variable).
1. Factor A affects the mean 2. Factor B affects the standard

deviation
B1
A1 A2
B2
Y Y
3. Factor C affects the mean and 4. Factor D has no effect

the standard deviation
C1
C2 D1 D2
Improve
1_05_01_004
Y Y
Role of Experimentation
The Master Black Belt is asked to define the role of experimentation by the Champion. He explains that a
statistically designed experiment is a test or a series of tests in which purposeful changes are made to the input
variables of a process or system so that we may observe and identify reasons for changes in the output
response. Once we have obtained the results of these tests, we can then decide how to change the settings or
levels of some factors in order to produce better yield, improved quality or some form of improvement of our CT
characteristic. The test results may lead us to select new factors which were dismissed or overlooked initially.
We would then design a new experiment, and so on, until we achieve the desired results.
The examples above imply that each factor can have independent effects on the result. We strive to find factors
which have independent effects like this, but this is not always possible.
The objective in many cases may be to develop a robust process, that is, a process affected minimally by the
many sources of variability, i.e.: white noise; which we cannot control.
The Master Black Belt concludes by saying that unless we experiment with our processes, guided by our
judgement, we will never identify ways to improve those processes. Experimentation provides us with a sound,
effective and economical method for determining the best way to change.
Sources of Experimental Factors
Input Factors (Xs) Output (Ys)

People Service
Material PROCESS
Equipment CTC
A blending of
Policies Product CTQ
inputs to
Procedures achieve the CTD
Methods desired Task
outputs
Environment
Improve
1_05_01_005
Sources of Experimental Factors
At a Six Sigma team meeting, a team member asks the Black Belt if he could help the team select factors for an
experiment on their process. The Black Belt points out that factors can come from a wide range of sources as
shown in the list above and that it is impossible to select the most influential factors unless you fully understand
the process. He suggests that they review the FMEA's that had been completed recently by the team, since
these contain an excellent checklist of what is important to this particular process and act as a summary of the
collective knowledge about the process from the brainstorming sessions.
The Black Belt also says that they need to be aware that factors can be continuous in nature (like volts,
pressure, etc.) or can be discrete (like new/old, machine 1/machine 2, etc.). Also, each factor needs to be
independently controllable.
The team now felt that they knew how to go about selecting factors for their first experiment.
Types of Experimental Factors
CONTROLLABLE INPUTS
Factors that we can control, and
elect to hold constant for the
experiment
OUTPUT (Y)
Response variable has
variation due to:
INPUTS (X’s) PROCESS • Effects:
Factors that we
wish to vary in a
UNDER – Factors
– Interactions
DOE STUDY
White Noise
• Experimental Error
– Measurement
– Other
• Environment
UNCONTROLLABLE INPUTS
Improve
Factors that we cannot control
1_05_01_006
• Environmental
• Unknown
Types of Experimental Factors
The team realizes that there are different types of factors involved in their process – some controllable and
others uncontrollable. They ask the Master Black Belt for advice in how to handle these.
The Master Black Belt draws the above chart on the board and describes a process he uses to ensure that all
factors are considered when designing a DOE. Before beginning, he reminds the team that a review of the
FMEA for their process may provide clues as to how to categorize the various factors.
"The first step", says the Master Black Belt, "is to divide all known factors into one of the three categories as
shown in the chart. Of all the possible inputs that could be used in a DOE, you need to decide which ones you
wish to vary in the experiment, and which ones you can move to the controllable (but hold constant) category. It
may be time well spent to put as many factors into this category as possible, especially ones that could be
varied, but are not likely to be allowed to vary for certification reasons, or just because it doesn't make sense.
Also, the factors that cannot be controlled are important to identify because there may be ways that their effect
can be reduced by careful design of the experiment, or recording of data that would otherwise have been
ignored or overlooked."
The Master Black Belt goes on to say that the input factors remaining are likely to be few (i.e. 2 to 5) so that the
experiment can focus on only these, resulting in as few runs as possible. He pointed out that, by the time this
exercise is finished, the experimenters will already be more knowledgeable about their process and be ready to
begin the design of their DOE.
What is "Design of Experiments" (DOE)

Definition
General
The controlled, active variation of a process to identify better settings or procedures to improve the
product quality.
Specific
A DOE is a systematic method where a number of factors are changed simultaneously, following a
predetermined pattern, to investigate their effect on the response or output.
What is "Design of Experiments" (DOE)
When the Black Belt is requested to define the concept of a DOE to the Champion, he uses the above definition.
As can be seen, this definition applies equally to transactional, engineering design, or to manufacturing
processes. The Black Belt explains that a DOE must have two basic elements – a design where the team
decides what to focus on, and a procedure for analyzing the data once it has been collected. He explains that a
DOE is a way to generate information about a process which can then be used to improve it.
The Champion says: "Tell me more, I am interested to see if I can apply this tool in the Customer Services area".
The Black Belt outlines other features of a DOE:
• It is a way of separating the vital few from the trivial many factors.
• It is a systematic scientific approach which allows decisions to be based on facts.
• It is a planned set of trials (runs) with specific measures, carefully recorded.
• It is a structured way to economically improve any process.
• It is dynamic, testing many factors (and interactions) in a short time.
In conclusion, the DOE process consists of a series of experiments where competing settings of each factor are
combined in order to efficiently reach valid and relevant conclusions.
The Shower Experiment
Flow Rate of Flow Rate of

Run Cold Water Hot Water
1 Low Low
2 High Low
3 Low High
4 High High
Improve
Y = f (X1, X2)
1_05_01_008
Water Temperature = f (flow of hot water, flow of cold water)
The Shower Experiment
During a refresher session on Design of Experiment (DOE), a Champion asks a Master Black Belt: "Do you have
a simple example to show us?". "Of course, we will use the shower example. The process is quite simple. By
adjusting hot and cold knobs, you will have optimal water temperature which represents the response of the
process."
"In most industries, we use two levels to do an experiment. In this case, let's suppose that the knobs can be set
only at high or low levels. The two knobs represent the two variables or factors of the experiment. "Then we
2
have a two-level, two-factor matrix, i.e. 2 , meaning that four combinations are possible to get results".
"We now can prepare the design matrix with factor Flow Rate of Cold Water and Flow Rate of Hot Water."
"It seems to be very easy, why don't we use this tool more often?" asks a Champion. "This is a very simple
example that we've done on the board and it represents a part of the planning of a DOE. Is it obvious to
determine which are the best settings? The experiment itself still needs to be done. Who wants to volunteer?"
asks the Master Black Belt.
"Remember, conducting an experiment requires human and material resources, parts, raw material, money,
time, etc. In order to get worthwhile results, each DOE should be carefully planned with all the required
resources" concludes the Master Black Belt.
Components of a DOE
1. Experimental Design Questions:

• Fixed Effects What result do we expect to get?
• Random Effects What are we going to measure?
• Mixed Effects
How will we collect the data?
How will we present the data?
Questions: 2. Analytical Procedure

How will we analyze the data? • ANOVA
How will we find relationships? • Regression

• Etc.
How will we find interactions?
Improve
How sophisticated does the
1_05_01_009
analysis have to be?
Components of a DOE
In his presentation describing the various features of a DOE, the Master Black Belt feels he should go into more
detail. He summarizes the questions that one needs to ask as shown above and explains the two components
as follows:
1. Experimental Design: A means to structure the variables for experimental manipulation. This can be broken
down into three types of effects.
• Fixed Effects
• The experimental levels are selected by the team.
• Conclusions apply only to selected factor levels.
• Most experiment designs are of this type.
• Random Effects
• Levels are randomly selected from a population.
• Conclusions can be generalized to the population.
• Mixed Effects
• A combination of fixed and random models.
2. Analytical Procedure: A means to extract relevant information from the experiment.

• Analysis of variance (ANOVA)
• Regression
• Find a relationship between variables.
In conclusion, the Master Black Belt states that, in the majority of cases, we use "Fixed Effects" designs and
ANOVA for the analysis procedure.
Why Not One-Factor-at-a-Time?
The autoclave operator insists that he should not vary more than one parameter at a time, fearing that the results
would be impossible to interpret. The Six Sigma Black Belt, trying to convince him otherwise, explains that since
there are so many factors (potential CTPs) involved in even simple processes, it would be very time consuming
and expensive to do a series of experiments "one-factor-at-a-time". He also points out how focusing on "one-
factor-at-a-time" will probably miss the optimum combination of factors.
The Black Belt illustrates this with a two-factor example. He explains that by varying the temperature (factor A), a
zone of maximum response is determined (1 above). Then by keeping temperature constant, the pressure
(factor B) is varied to determine its best response (2 above). One may conclude that this process defines the
optimum settings, but if we overlay this picture onto the real situation (3 above), the so-called optimum is far from
defining the maximum potential value (4 above). As one can imagine, this becomes more pronounced when
more variables are involved.
A much better approach is what is termed a "full factorial design" where a few factors are varied at the same
time, in random order, to yield a matrix of effects. This approach includes interaction effects which can be very
important in some cases, and would be missed completely in the "one-factor-at-a-time" approach.
The Black Belt sits down with the autoclave operator and the Six Sigma team to design such an experiment.
They are anxious to try a DOE because they feel that there are significant improvements that can be achieved.
Benefits of Statistical Experiments
Change
One
Evaluation Expert Everything Factor Stat
criteria Observe Guess Opinion at once At-a-time DOE
Time to Very Long

get results Long Long Medium Long Overall Short
Very High Rel

Cost High High Medium High Overall Low
Chance of Sub-
detecting Very optimum Very
optimum Low Low Moderate Moderate likely Good
Confidence Very Very

in results Low Low Moderate Low High High
Ability to
identify
independent
interactions
and main
effects None None None None None Yes
Conclusion: Statistical Experiments are Rich With Information!!
Benefits of Statistical Experiments
A Black Belt observes over the course of the past year that there is a lot of folklore about the benefits of
observation, judgement and "one-factor-at-a-time" as techniques for deciding how to make a change to a
process. He decides to map the various techniques and to identify their strengths and weaknesses compared to
statistical experiments. The above table is the result of his efforts. Using a subjective scale, one can see that, for
all criteria selected, the statistical experiment is far superior than any of the others.
After creating the table, the Black Belt makes some notes in the way of conclusions:
• Statistical experiments provide fast, low cost, optimal results with a high degree of confidence. However, in
the short-term, they do cost money and disrupt the process, so they must be properly planned.
• In addition, statistical experiments are the only technique to provide the data for "main effects" and
"interactions" between various factors.
• We use experiments to supplement our judgement, not replace it.
• Properly designed experiments will permit simple interpretation of results, which may not be possible
otherwise.
• Proper experimental design is considered to be more important than sophisticated statistical analysis.
• The cost of a DOE is much lower than the cost of poor quality (long-term).
Types of Experiments
Very • Trial and Error Methods

Informal – Introduce a Change and See What Happens
• Running Special Lots or Batches
Traditional
Approach
– Produced Under Controlled Conditions
• Pilot Runs
– Set up to Produce a Desired Effect
• One-factor-at-a-Time Experiments
– Vary One Factor and Keep All Other Factors Constant
• Planned Comparisons of Two Methods
– Background Variables Considered in Plan
• Experiment Planned With Two to Four Factors
Six Sigma
Approach
– Study Separate Effects and Interactions
• Experiment With 5 to 20 Factors
– Screening Studies
Very • Comprehensive Experimental Plan With Many Phases
Improve
1_05_01_012
Formal – Modeling, Multiple Factor Levels, Optimization
Types of Experiments
After having some exposure to the basic concept of a DOE, a Black Belt in training asks how the traditional
approach that he had been familiar with compares to the Six Sigma approach. "Good question," replied the
Instructor, "let me write out what I'll call the "hierarchy" of experiment types." He writes down the first four
headings and everyone agrees that they know what these involve. He went on to write the headings that
describe the Six Sigma approach. At this point in the course, the students are not familiar with these, but they
are amazed to think that up to 20 factors can be dealt with within one experiment.
The Black Belt in training asks: "Does this mean that we will no longer use the traditional types of experiments?".
The Instructor replies that they will still be used on occasion – a special lot or a pilot run could still be used on
occasion if you were wanting to check the data collection process for calibration purposes, etc. He said that once
everyone had some experience with the Six Sigma approach, they would readily see that it gives much better
information than the traditional approach and that the new tools would be easier to use.
Classes of DOE
Screening Characterization Optimization
A E B
C B D K K
Factors I K F F
F J J
H G
Time
Improve
1_05_01_013
Classes of DOE
There are several classes of DOE that can be applied to improve a process. When a team member asks the
Master Black Belt to give an overview of how the various classes of DOE are applied, he draws the above chart
and elaborates on each as follows:
• Screening designs are generally used to identify the "vital few" process factors, putting the "trivial many"
aside. Usually, such designs involve a large number of factors and are limited to only two levels.
• Characterization designs are generally used to study simple effects involving a small number of factors (3 or
4). Such designs are usually based on full factorial models using two or three levels.
• Optimization designs (also referred to a modeling designs) are used to study complicated effects and
interactions involving one or two factors. In general, such designs are employed to yield a precise
mathematical model of the process under investigation and may involve several levels (to capture non-
linearities).
The Black Belt points out that the above class descriptions are very general in nature and should not be used as
the sole benchmark when designing an experiment. In practice, more than one experiment is generally required
to obtain a meaningful model, the results of one being used to design the next. Of course, each experiment is
different from the ones before, being tailor-designed for the particular factor and levels being investigated. He
also points out that, most often, we work with screening designs, occasionally with characterization designs, and
rarely with optimization designs.
Terms Used in DOE
Process Coded
Factor Variable Setting Value
A Pressure High (40psi) +1 Y = f (A, B, C)
Low (20 psi) -1
B Machine New +1 Treatment or
Old -1 Factors Combination
C Shift Number 1 +1
Number 2 -1 Sequence Process Variables Response
Order A B C Variable
1 -1 -1 -1 5.4
Order
2 +1 -1 -1 6.1
3 -1 +1 -1 7.2
Data 4 +1 +1 -1 3.7
5 -1 -1 +1 9.3
6 +1 -1 +1 5.5
Improve
Levels 7 -1 +1 +1 8.3
1_05_01_014
8 +1 +1 +1 10.7
Terms Used in DOE
As a means of introducing the topic of DOE to the Six Sigma team, the Black Belt presents the above chart
which represents the design anatomy of a typical experiment. He is asked to review the primary terms, which he
does as follows:
• DOE – Design of Experiment, or "an invitation for an information event to occur".

• Order – Sequence order or run order for the experiment.
• Factor – Process variable that will be purposely altered during the experiment.
• Level – Factor setting.
• Treatment Combination – Combination of levels for this element of the experiment.
• High Level – High setting (e.g.: 40 psi, new, shift 1, etc.).
• Low Level – Low setting (e.g.: 20 psi, old, shift 2, etc.).
• Response Variable – The subject of the experiment (CTQ, CTD, CTC).
• Data – The result – i.e. the magnitude of the response variable.
The Black Belt points out that the matrix shown is in the form convenient for the design of the experiment, and
that later on we will see how to arrange this data so the experiment can be conducted.
From this brief description, each team member has a clear understanding of the design structure of a DOE and
begins to plan an experiment that applies to their process.
Main Effects and Interactions
Y B- Condition
Run A B A*B Y 5
∆=2
1 -1 -1 1 3.0 3
2 1 -1 -1 5.0
A- A+
3 -1 1 -1 5.0
Y B+ Condition
4 1 1 1 12.0
12
#1 #2
∆=7
Main Interaction
Effect Effect 5
Improve
1_05_01_015
A- A+
Main Effects and Interactions
Before beginning to design their experiment, the team leader asks the Black Belt to explain what is meant by
"main effect" and "interaction". The Black Belt uses the above example to describe the two terms.
Main effect describes the direct effect of factor A on the response variable (Y). Likewise, B is another main
effect. Factors A and B are generally assumed to be totally independent of one another (2 vs. 7).
Interaction effect describes the effect of the product A x B on the response variable (Y) when two factors have a
combined effect on the response variable.
The Black Belt points out that one of the powerful advantages of a well designed DOE is that it has the benefit of
being able to identify interactions which would be totally missed in a one-factor-at-a-time experiment. The
interaction effects can be used to achieve the desired result. The two charts on the right hand side above show
the interaction effect, i.e.: when B is high, the yield (Y) of the process is over three times better than when B is
low.
Contrast - Definition
Contrast of factor F = ΣΥ F+ - ΣΥF-
DESIGN MATRIX Vectored

Vectored
Column
Column
Response
Response
Run Order A B AB Y A Y
1 (1) -1 -1 1 1.5 -1 * 1.5 = -1.5
2 a 1 -1 -1 4.5 1 * 4.5 = 4.5
3 b -1 1 -1 4.5 -1 * 4.5 = -4.5
4 ab 1 1 1 13.5 1 * 13.5 = 13.5
12.0
Example Computation
Column
Run Order A B AB Y1
Contrast
1 (1) -1 -1 1 1.5 Sum of Vectored
Responses
2 a 1 -1 -1 4.5
3 b -1 1 -1 4.5
4 ab 1 1 1 13.5
Improve
Contrast 12.0
1_05_01_016
Contrast – Definition
During a training session on Design of Experiment (DOE), a Black Belt asks: "You mentioned the word
CONTRAST in your explanation, but I don't understand what it means". The Master Black Belt responds: "Actual
factor settings for an experiment, such as Low, High, 80 psi, 120 psi, etc, are replaced by coded values (+1, -1)
called coefficients in a design matrix. The coefficients of a factor or interaction are used to calculate its contrast.
To define a contrast, a set of coefficients have to conform to conditions known as "balanced" and "orthogonality",
which we will learn later."
"As you can see on the visual, the contrast of one factor or interaction is the result of the sum of the coefficients
multiplied by the response of each run. In other words, we can say that a column contrast is the linear sum of
vectored responses."
"Thus, for a main effect, a positive contrast means that the response variable is greater when the factor is set at
its high level (+1), compared to the low level (-1). A negative result means the inverse, i.e. that the response
variable is greater when the factor is set at its low level (-1), compared to the high level (+1). Then contrast
compares two things or groups of things by comparing their difference. It is just a special way of making a
comparison."
"Is it then accurate to state that the term contrast is used for more than one thing?" asks a Black Belt. "Yes,
actually contrast is an intermediate calculation used in statistical analysis of a DOE, and it also defines a set of
coefficients used in a design matrix" concludes the Master Black Belt.
Contrast - Properties
DESIGN MATRIX
Run A B AB
1 -1 -1 1
2 1 -1 -1
3 -1 1 -1
4 1 1 1
Balanced ΣX i = 0 for each factor sum
Improve
Orthogonal Σ Xi X j =
1_05_01_017
0 for all dot product pairs
Contrast – Properties
During a training session on Design of Experiment (DOE), a Master Black Belt says: "We know that a column of
a design matrix is considered as a contrast under certain conditions, but do you know what these conditions or
properties are"?
A Black Belt responds: "Balanced and orthogonality, but I don't have a clue as to what these mean". The Master
Black Belt says: "Well, take a design matrix for example. We have factors and interactions between factors. To
be considered as a contrast, the sum of a set of coefficients must equals zero. This is the basic condition to have
contrast. When the sum of each column equals zero, the experiment is said to be balanced. In a case of a two-
level design, this condition means that for one column, there is the same number of High (+1) and the Low (-1)
levels."
"When you closely look at the picture, we see that the AB interaction is created by multiplying the coefficients of
factor A by the coefficients of factor B, and the sum of the coefficients of AB contrast is also equals to zero. For
every pair of contrasts, when the sum of the product equals zero, then the design matrix is said to be orthogonal.
This feature ensures the effects are independent. Because all effects can be independently estimated, small
effects will not be washed out or masked by large effects. Actually, you don't have to build orthogonal contrast.
Minitab software will automatically provide a balanced and orthogonal designs" concludes the Master Black Belt.
Run Order for a DOE
Randomized Design
RunOrder Order Factor A Factor B Factor C
1 16 1 1 1
2 15 -1 1 1
3 10 1 -1 -1
4 11 -1 1 -1
5 3 -1 1 -1
6 5 -1 -1 1
7 13 -1 -1 1
8 8 1 1 1
9 12 1 1 -1
10 11 -1 -1 -1
11 4 1 -1 1
12 6 1 -1 1
13 2 1 -1 -1
14 9 -1 -1 -1
Improve
15 7 -1 1 1
1_05_01_018
16 4 1 1 -1
Run Order for a DOE
Once the factors and levels to be investigated are identified, you must then decide how many replications you
wish to do. The use of Minitab makes it easy to lay out how to actually run the experiment. For a three-factor,
two-level experiment, with two replications, the above table shows how the run sequence would appear.
The Black Belt states: "You should note that the original order from the design matrix is now randomized. The
original design order is given in the second column. If we wanted to have three replications instead of two, the
list of runs would be increased by 8 and the whole list would then be randomized."
A team member asks: "Why is it necessary to randomize the run sequence?". The Black Belt responds by
saying: "Running the experimental trials in a random order protects against the effects of unknown non-
experimental variables (like environment) that might bias the results. In other words, if some unknown factor is
changing continuously over the period of the experiment, its influence will be minimized by this technique."
How to Select Factor Levels
80
Response Variable (Y)

70
60
B difference
50
A difference
40
30
B- high
A-high
B-low
A-low
20
10
0 100 200 300 400 500 600
Factor Settings
Y Y
True Effect True Effect
Improve
Experimental Effect Experimental Effect
1_05_01_019
Lo Hi Lo Hi
Factor Settings Factor Settings
(-) (+) (-) (+)
How to Select Factor Levels
A Black Belt makes the statement that selecting factor levels is a critical step in determining the success of a
DOE. "If that's true," said the Champion, "can you help us with a criteria to do this?". "Yes," said the Black Belt,
"but remember, it is much better to plan a series of experiments from the beginning than it is to try to do it in only
one. You should plan to spend only 25% of your allotted time and budget on your first experiment." The Black
Belt outlines the important aspects as follows:
• Draw on all past experience to provide knowledge of your process.

• The intent is to capture a large meaningful result, well beyond white noise or measurement resolution.
• Select step sizes to ensure not missing the optimum.
• Levels for each factor should be balanced in relation to each other (i.e.: no one dominant).
• Levels need to be logical, rational, practical, and safe.
• Initial levels should be bold or wide, reducing to narrower steps in subsequent experiments.
• A good rule of thumb is 20% beyond input spec limits.
• If no spec limits exist, use ± 3 standard deviations to set levels.
• Use a pilot run to verify sensitivity to level selections.
Confounding Effects
MATRIX A
Coded values Printer Paper
Printer 1 Printer2
-1 #1 Low Quality
Paper Type Low Paper Type High
1 #2 High Quality
ST
AN
DA
RD
MATRIX B
Run Printer 1 Paper
1 -1 -1
2 1 -1
Improve
3 -1 1
1_05_01_020
4 1 1
Confounding Effects
A Black Belt is working on a project to minimize the number of rejected reports due to bad printing quality that
occur with the two main printers located in the computer room. After a rigorous study, the factor "quality of
paper" will be analyzed. During a coaching session with a Master Black Belt, the Black Belt proposes to conduct
a Design of Experiment (DOE).
"What is the objective of the experiment?" asks the Master Black Belt. The Black Belt says "We want to know the
effects of the "quality of paper" on printing quality. In this case, we hypothesized that an increase in "quality of
paper" increases the quality of printing. To test this theory, we did the design matrix A initially, but in this case it
shows all low settings for printer 1 and high settings for printer 2. I fear that the differences in the response
variable due to "quality of paper" won't be separated from the unique effects of the printer. In other words, the
effect due to "quality of paper" will be confounded or mixed in with the printer in use." The Black Belt asks the
Master Black Belt for advice.
"If we design a new matrix as shown in B with both types of paper in both printers, we will be able to differentiate
the contribution of each paper type in the response variable, assuming that the influence of all other background
variables have been adequately controlled or otherwise, neutralized" says the Master Black Belt.
"Yes, I see that by doing this, we will be able to draw meaningful conclusion from the collected data" concludes
the Black Belt.
Strategy of Experimentation
1. Define the Problem
2. Establish the Objective
3. Select the Response Variable(s)
4. Select the independent factor(s)
5. Choose the factor levels
6. Select the Experimental Design
7. Collect the Data
8. Analyze the Data
9. Draw Conclusions
10. Achieve the Objective
"The purpose of an experiment is to better understand the real world, not to understand the
experimental data."
William Diamond
IBM – Retired Statistician
Strategy of Experimentation
A Champion asks the Master Black Belt to list the general steps involved in conducting a Design of Experiment
(DOE).
"Well, after the Measure and Analyze phases, we should define and therefore state the problem. For example,
the mean of the process under study is not on target or there is too much variation, and as result we are
producing a large amount of rework. The objective is to reduce the PPM defective to avoid rework. At this phase,
we select the independent variables that we want to test, the X1, X2, …, XN. Now, we are ready to select
appropriate levels of each factor knowing what would make sense for our process. We then assign these levels
-1 and +1 notation."
"When do we start collecting data?" asks a Champion. "One more step is required before collecting data" replies
the Master Black Belt. "The selection of the experiment design, or the structure of the experiment i.e. Full
Factorial, Fractional Factorial, etc."
"All of these preceding steps constitute the planning part of an experiment. An experiment design is the
deliberate arrangement of competing settings among one or more independent variables, and when properly
planned or designed, it will permit relatively simple interpretation of the results, which may not be possible
otherwise."
"I'm impressed," says the Champion, "you seem to be very comfortable with this subject. If you don't mind, I
would like to assist you in conducting one experiment, so I will learn more about this tool and be able to provide
more assistance to the Black Belts".
Barriers to Effective Experimentation
At the Annual Six Sigma Symposium, a guest speaker presents the powerful tool "Design of Experiment" (DOE).
He describes all the advantages and steps to conduct a valuable experiment. During the question period, an
Black Belt asks: "I see that there are many advantages to using this tool, but what are the difficulties or obstacles
that we should be aware of before using it?"
"This is a good question", replies the guest speaker, "let me show you a list of the main barriers to effective
experimentation. As we can see, eliminating some of these barriers is within your reach. Why should we conduct
a DOE if the problem definition or the objectives are not clear? Some of these barriers may be attributable to a
lack of training and information. Ask why there are negative perceptions in your organization? You are deploying
efforts but do you adequately communicate your plan and progress to your organization? Think about it".
"As in any organization, because of a lack of time and because of other priorities, management may not always
be able to provide you with adequate support and coaching. You have to involve them in the realization of a
DOE, so that they will start to learn, understand and promote the use of these powerful tools. But once again, I
insist on the importance of carefully planning your DOE. A good plan, which is adequately communicated may
prevent negative perceptions and promote management's active involvement. With perseverance, we can initiate
change even in an environment of excessive bureaucracy. My final message to you is DON'T GIVE UP", says
the guest speaker.
The Car Mileage Experiment
Experiment Design:
Objective: How to improve Factor Level 1 (-) Level 2 (+)
fuel economy on A. Type of Gasoline Regular High Octane
the highway. B. Cruise Speed 100 Km/hr 130 Km/hr
C. Tire Pressure 20psi 30psi
D. Windows Open Closed
Measure : Litres/100 Km. E. Air Conditioner Off On
Results:
Factor (in Order Best Factor
Of Importance) Level
B. Cruise Speed 100 Km/hr
DE. Windows * A.C. Windows Closed/A.C. Off
C. Tire Pressure 30psi
D. Windows Closed
Improve
1_05_01_023
E. Air Conditioner Off

A. Type of Gasoline High Octane
The Car Mileage Experiment
A Master Black Belt conducts a DOE on his automobile to try to understand which factors are important
contributors to fuel economy. During a training session, he presents the results to a class of Black Belts in
training. He makes the following points as a way of illustrating the various aspects that need to be considered in
any DOE.
• This one relatively simple experiment produced a wealth of knowledge.

• The actual order of importance of the various factors is contrary to his original thoughts (order in the first box
above).
• An interaction between the windows and the air conditioner proves to be the second largest effect (not even
imagined).
• What probably really matters is minimizing cost, not maximizing fuel economy. If, for instance, the type of
gasoline had been more significant, one needs to include the extra cost for super grade to determine if it is
still an advantage.
• Before concluding that the type of gasoline, for instance, is significant, one needs to ensure that the ability to
measure fuel mileage is accurate enough to differentiate such small effects (i.e.: is measurement error
causing a false conclusion).
• There are generally many other factors involved. These must be kept constant as much as possible in order
to obtain valid results.
One Factor Analysis
Re sponse Fa ctor
0.99265 1
ab
1.06742
it0.87233
1
1
in 1.17466 1
M 0.88709 1
0.79582 2
1.12497 2
1.27382 2
1.20943 2
Level 1 Level 2 Level 3 1.00341 2
Wilsons Acme Twecho Factor 1.70682 3
0.99265 0.79582 1.70682 1.63187 3
1.06742 1.12497 1.63187 1.70582 3

1.61013 3
0.87233 1.27382 1.70582 Response 1.27879 3
1.17466 1.20943 1.61013
0.88709 1.00341 1.27879
Improve
1_05_01_024
One Factor Analysis
A printed circuit board is related to high field failure rates. Following an analysis of the problem a cross-functional
team concludes that the primary failure is due to low adhesion of the circuitry. After careful consideration of the
evidence the team hypothesized that the problem is related to the type of solder paste used during production.
Three types of paste are available; Wilsons, Acme, and Twecho. An Engineer on the team is tasked with
evaluating the pastes to determine which one is the best. To make this evaluation he randomly selects 15
printed wiring boards and randomly assigns them to the three paste types. After curing he subjects each board
to a tensile test to quantify the adhesion of the different pastes.
At a team meeting the Engineer gave the team some general information about the single factor analysis which
he was conducting. "The dependent variable is called the response. It must be measured on an interval or ratio
scale; in our case the measurements will be in pounds. The independent variable is commonly known as a
factor. In all cases the factor must be treated as a categorical variable. Our factor, of course, is the paste.
Continuous factors must be classified into subgroups or levels. If the factor is discrete, as in our case, further
classification may not be required as the subgroups naturally exist."
"Now that you have defined the factor levels and have measured each response, how will you analyze this
data?" one team member asked.
"We will look at the influence of the different levels both practically and statistically. To do this we will enter the
data in Minitab, stack it, conduct a one-way ANOVA, analyze the sum of squares, create a main effects plot, and
do a residual analysis."
Sum of Squares As Related To

One-Way ANOVA
Analysis of Variance
Source DF SS MS F P
cal
i
Factor 2 1.0127 0.5064 18.24 0.000
ti st
Error 12 0.3331 0.0278 a
Total 14 1.3458 St
Level 1 Level 2 Level 3

Wilsons Acme Twecho
0.99265 0.79582 1.70682
1.06742 1.12497 1.63187
0.87233 1.27382 1.70582
1.17466 1.20943 1.61013
0.88709 1.00341 1.27879
0.016 0.036 0.032 Variance Column

l
0.064 0.143
t i ca 0.126 SS Column
75%
r ac 1.01 SS Between (Source)
Improve
P
1_05_01_025
25% 0.33 SS Within (Error)

1.35 SS Total (Total)
Sum of Squares as Related to One-Way ANOVA
An Engineer was evaluating three brands of soldering paste used on printed circuit boards. Fifteen tensile
strength measurements were taken; five for each of the different types. To estimate the practical significance,
the Engineer calculated the sum of squares within (SSW), between (SSB), and total (SST). By adding the SS for
each of the levels (0.64 + 0.143 + 0.126) he established the SSW (0.33), this represented the error. By
combining the response data for all of the levels he calculated SST. Next he subtracted the SSW from SST
which resulted in the SSB. The SSB represented the source of variation or the variation attributable to the factor
under study. He calculated a ratio of the SST compared to SSB and the SSW and found that the SSB amounted
to 75% of the variation and SSW was attributable to 25% of the total variation. He concluded that the difference
in the levels was an appreciable influence of variation. He reviewed his findings with the Black Belt.
After reviewing the Engineer's analysis the Black Belt remarked, "Your analysis has revealed the practical
influence of the factor under study. To better understand the implications of the differences you have uncovered
we need to look at the data statistically." The Black Belt entered the data into Minitab, completed a one-way
analysis of variance (one-way ANOVA) and reviewed the results with the Engineer. "The one-way ANOVA also
calculates the SS within, between and total. Based on the SS values a mean square (MS) ratio, which is the F
value, is derived. This report also generates a P value which also indicates that there is a statistically significant
difference (P < 0.05) in at least two of the means differ from the rest. These values expose the statistical
influence of the different levels. The practical influence of different levels can also be exposed using Minitab by
taking the SS and calculating a ratio as we did in Excel. The practical can then be displayed using charts such
as a pie chart."
One-Way ANOVA Table
An Engineer is working on a project to maximize the tensile strength of the bond of circuitry on a printed circuit
board. Three different types of paste were suggested as possible solutions to the low tensile strength problem.
Fifteen measurements were taken, 5 on each type of paste. The Engineer was unsure how to analyze the data
and asked the Black Belt for help.
After reviewing the data, the Black Belt suggested two possible analysis tools, the t test and the one-way
ANOVA. "In Minitab we can run both of these tests. However, ANOVA has the advantage of comparing various
levels, while the t test makes paired comparisons. It appears that the Twecho paste is producing a significantly
higher tensile strength." "How do you know this?" the Engineer asked.
"The one-way ANOVA output in Minitab produces a graphical representation of the mean confidence intervals
and an F statistic and a P value. We can see that the confidence interval around the mean of the Twecho paste
is completely outside the range of the confidence interval of the other two paste.
The use of ANOVA has some underlying assumptions of the data; the population variances of the response are
equal across all levels for a given factor, and the residuals are constant and normally distributed with a mean of
zero and a constant, but finite variance." "How do I analyze the residuals?" the Engineer asked. "This can also
be done in Minitab with the residual analysis tool."
One-Way Main Effects Plot
Level 1 Level 2 Level 3

Wilsons Acme Twecho Main Effects Plot - Means for Response
0.99265 0.79582 1.70682
1.06742 1.12497 1.63187
0.87233 1.27382 1.70582
1.17466 1.20943 1.61013
1.6
0.88709 1.00341 1.27879
1.5
1.4
Response
1.3
1.2
1.1
1.0
1 2 3
Improve
1_05_01_027
Factor

One-Way Main Effects Plot
An Engineer is working on a project to maximize the tensile strength of the bond of the circuitry to a printed
circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength
problem. Fifteen measurements were taken, 5 on each type of paste. To analyze the data the engineer created a
one-way main effects plot. On the chart he depicted the average response for the three different levels, that is,
the three different types of paste. Having completed the analysis the Engineer presented the results to the
project Black Belt.
"As you can see by this main effects plot the paste at level three provides us with the maximum tensile strength
of the three types; this is the type that will be specified from now on."
"This looks very good and seems to support the selection that you have made. However, do you have statistical
evidence to support this selection?"
The Black Belt asked. "Statistical evidence?"
The Engineer asked. "The main effect plot is a very good graphical tool to help substantiate your conclusions;
however, it should be used to compliment other tools. With this tool alone I can't be sure that the variation
presented in the plot is significant. I need to know with a certain degree of confidence if the differences between
the averages could have happened as a result of chance random sampling variation or not. To do this we need
to first use tools such as one-way ANOVA."
Residual Model Diagnostics

Normal Plot of Residuals I Chart of Residuals
4 10
3 3.0SL=7.536
2
1
Residual
Residual
0
0 X=0.000
-1
-2
-3
-4 -3.0SL=-7.536
-5 -10
-2 -1 0 1 2 0 5 10 15
Normal Score Observation Number
Histogram of Residuals Residuals vs. Fits

4 4
3
3 2
Frequency
1
Residual
0
2
-1
-2
1 -3
-4
0 -5
-5 -4 -3 -2 -1 0 1 2 3 4 10 11 12 13 14
Residual Fit
Residual Analysis
An Engineer was performing a one-way ANOVA comparing three different types of soldering paste. The
Engineer remembered from his training that one of the underlying assumptions of the ANOVA is that the
residuals are random and normally distributed. However, he did not know how to verify this. He asked the project
Black Belt for help.
The Black Belt explained. "When you are doing your one-way ANOVA in Minitab you need to store both the
data's fits and residuals. This data is then used to construct a residuals plot."
Together they generated a Residual Model Diagnostics plot. The Black Belt reviewed the plot and proclaimed,
"Very good, it appears that your assumptions have been meet."
"How can you tell?" the Engineer asked.
"The two left charts, the Normal Plot of Residuals and the Histogram of Residuals, both give an indication of the
normality of the residuals; both look normal. The I Chart for Residuals gives us an indication of change in the
residuals through time. This chart implies that the points are randomly distributed through time; that is, the points
are evenly distributed above and below the central line, most points fall near the central line, some points
approach the control limits, and no points are beyond the control limits. The last chart, Residuals vs. Fits, gives
us evidence of changes in residual variation at the different levels. Each group appears to be similar in
variance."
The Two-Level Two-Factor (2 2)
Design - 4 Representations
Geometric
HIGH A
Standard
Low High
B Interior Space
Low (1) a
B
LOW HIGH
A High b ab
Tree Algebraic
Low
Run A B
A Order
High 1 -1 -1
Design 2 +1 -1
Improve
Low 3 -1 +1
1_05_01_029
B 4 +1 +1
High
The Two-Level Two-Factor (22) Design – 4 Representations
"A two-factor, two-level design can be represented in at least four different ways" says a Master Black Belt to a
Champion.
In the "Geometric" representation, the corners of the square represent the levels of factors A and B, from "low-
low", to "high-high".
The "Standard", or Yates designation (in the upper right image), shows the notation of all treatments taken from
the treatment combination. "a" indicates "A high" and "B low", "b" indicates "B high" and "A low". "ab" indicates
that both factors (A and B) are high. The notation used in the Yates designation refers to the factor set at the
high level, except for the case of (1), which refers to both factors (A and B) being set at the low level.
The "Tree", also called "Taxonomy" or "Characterization Chart", illustrates the experiment design for factors A
and B, with their respective levels (low and high).
The fourth image, called "Algebraic" form is used to represent the settings used for each run (from 1 to 4), with a
plus (+1) or minus (-1) sign to denote if the factors are set to their high or low settings.
The Champion recognizes that the "Algebraic" and "Yates Standard Notation" are directly related to the runs of
the experiment. The Master Black Belt adds: "the "Algebraic" notation respects Yates Standard Order, and is the
most common representation of full factorial experiments".
Other Factorial Design Notation

Matrix (22)
Two-Level Two-factor Design
Sign Alternative Japanese Classical

Notation Notation Notation Notation
Run No. A B A B A B A B
1 - - 0 0 1 1 (1)
2 + - 1 0 3 1 a
3 - + 0 1 1 2 b
4 + + 1 1 2 2 ab
Improve
1_05_01_030
Other Factorial Design Notation Matrix (22)
A Black Belt asks her Master Black Belt: "In our DOE, we always identify individual trials with –1 or 1 . I have just
read a book where the author is using another designation. Is there any difference and advantages in using one
compared to the others?"
k
The Master Black Belt replies: "Remember, the 2 factorial designs, i.e. a factorial experiment with k factors,
k
each at two levels, has widespread industrial applicability. The experiment consists of 2 trials, one trial at each
combination of levels of the factors. To identify the individual trials, different notations are used. One convention
is to label each factor by a letter (or numeral) and then to denote the two levels (versions) of each factor by a
plus (+) and a minus (-) sign. Commonly the minus sign refers to lower level, the standard condition, or absence
of the factor (e.g. not lubricated vs. lubricated bearing). Thus if there are two factors at two levels, labeled A and
2
B, the four trials comprising 2 factorial design can be represented as shown in this table."
"…You are familiar with the –1, 1 notation used in Minitab, which is similar to the sign notation. Alternative
notations are to employ 0 and 1, respectively, or, following the Japanese tradition earlier established by Taguchi,
1 and 2 for the two version of each factor. The classical convention is to denote the two versions of each factor
by the presence and absence of its corresponding lowercase letter, here the trial in which all factors are at their
"low" level is denoted by (1)."
The Master Black Belt adds: "The convention we have adopted is to use the -1, 1 notation, which makes it
possible to identify, factor levels and contrast coefficients used to calculate factor and interaction contrasts when
performing DOE analysis."
Yates Standard Order (2 2)
Quenching Speed = A
Low: 1 m/s = -1
Yates Standard Order Table (2 3)
High: 2 m/s = +1
m/s = meter per second Factors
Aging Time = B
Low: 4min = -1
Run Treatment
High: 8min = +1 Order combination A B C
Bath = C
Low: Water = -1
1 (1) -1 -1 -1
High: Glycol = +1 2 a +1 -1 -1
3 b -1 +1 -1
Factors 4 ab +1 +1 -1
A 5 c -1 -1 +1
Run B C
Order Quenching
Aging Time Bath
Speed 6 ac +1 -1 +1
1 1 m/s 4 min Water 7 bc -1 +1 +1
2 2 m/s 4 min Water 8 abc +1 +1 +1
3 1 m/s 8 min Water
4 2 m/s 8 min Water
5 1 m/s 4 min Glycol
Improve
6 2 m/s 4 min Glycol Settings for a two-level
1_05_01_031
7 1 m/s 8 min Glycol

8 2 m/s 8 min Glycol two-factor design
Yates Standard Order (22)
A Six Sigma team conducts an experiment in the heat treatment process. Three factors are identified:
Quenching speed (A), Aging time (B), and Type of bath (C).
The levels of each factor are set as follows:
• Quenching speed (factor A): Low: 1 m/s (-1) High: 2 m/s (+1)
• Aging time(factor B): Low: 4 min (-1) High: 8 min (+1)
• Bath type (factor C): Low: water (-1) High: Glycol (+1)
"When designing the experiment, the Yates Standard order is the one most often used", says the Master Black
Belt. We used a two-level, three-factor design. For each run, the corresponding factor level is identified as -1
when the factor is set at "low", and as +1 when it is set at "high".
The first combination shown in the Yates Standard Order will be a treatment with Quenching speed set at -1
(low, or 1 m/s), Aging time at -1 (low, or 4 min.), and Type of bath set at -1 (low, or water). This run is
th
represented in the first row by the numbers: -1, -1, -1. All other runs are set in the same fashion, up to the 8
run where the settings are +1, +1, +1. The Master Black Belt explains that it is important to randomize the Yates
Standard Order prior any experiment in order to average out the noise caused by nuisance variables if they
exist.
The Master Black Belt explains that the same table can be used to plan an experiment for a two-level, two-factor
design. The team recognizes that using this Yates Standard Order can help them avoid errors in preparing an
experiment.
Computing Contrasts and Effects

(22)
Specifications
Ohm’s Law Factor Low High
Voltage = Resistance x Amperage Response Y: V Maximize -1 +1
Factor A: 6.0Ω ±50% A 3.0 9.0
Uv = RΩ * I A
Factor B: 1.0A ±50% B 0.5 1.5
Y = A*B
Y
Main Effect Plot for B
9.0
Run Order A B AB Y1
1 (1) -1 -1 1 1.5
2 a 1 -1 -1 4.5
3 b -1 1 -1 4.5
3.0
4 ab 1 1 1 13.5
Contrast 12.0 12.0 6.0
Effect 6.0 6.0 3.0
Avg + 9.0 9.0 7.5 A and AB Factor B
-1 +1
effects
Improve
Avg - 3.0 3.0 4.5 B - B+
1_05_01_032
∆ 6.0 6.0 3.0

are neutralized (A- & A +) (A- & A +)
(AB - & AB +) (AB - & AB +)
Computing Contrasts and Effects (22)
A Six Sigma team conducts an experiment. Two factors are considered (Resistance and Amperage), and the
objective is to maximize output (voltage). Resistance (factor A), is specified at 6.0 ? ? 50%, and Amperage
(factor B) at 1.0 A ? 50%. The team defines the factor settings at Alow = 3.0 ?, Ahigh = 9.0 ? and Blow = 0.5 A
and Bhigh = 1.5A.
The experiment is designed according to Yates Standard Order for one replicate only, and the output (voltage) is
recorded for each run (e.g. Run #3: Voltage equals 4.5 V for a Resistance of 3.0 ? and an Amperage of 1.5 A).
The Black Belt computes the contrasts for each factor (e.g. 12 for factor A), and the corresponding effects (6.0
for both factor A and B). Similarly, he calculates the effect for the interaction of the two factors (3.0 for the
interaction AB). The Black Belt explains that since the effect of B is equal to +6.0, the voltage (output) increases
by an average of 6 volts when the amperage (factor B) increases from low to high. Since the "B effect" is
positive, the voltage (output) will vary in the same direction as the direction of change of factor B.
The team concludes that to maximize voltage (output), Amperage (factor B) must be kept at the high level. The
Black Belt highlights the fact that with this full factorial design, each contrast (A, B or AB), represents the effect of
only one particular factor or interaction since the other two are neutralized by the combination of + and – signs.
In other words, the contrast of A represents variation due only to factor A since B and AB are neutralized. Finally,
the Black Belt reminds the team of the characteristics of the contrasts: a) Balanced, and b) Orthogonal, to
ensure independence of effects.
Computing the Sums of Squares and Practical Significance (22)
"Why do we calculate the contrasts"? Ask a team member. "Contrasts are part of the foundation of Analysis of
Variance", replies the Black Belt. They make it possible to compute the effects and the Sums of Squares. In our
two-level, two-factor design, the contrasts for factors A and B, along with their interaction AB allow us to compute
the Sums of Squares. In our experiment we see that contrast A is positive, meaning that the output is higher
when we set factor A at the high level compared to when it is set at the low level."
"What is the purpose of the pie chart?", asks another team member. "This chart, created from the Sums-of
Squares in the ANOVA table, shows the practical significance of the sources of variation" replies the Black Belt.
"In our experiment, SSfactor A accounts for 52%, SSfactor B for 38% and SSinteraction A*B for 10% of the
SStotal (i.e. the total change in the response variable. We can also conclude that 90% of the variation of the
response variable (CT Characteristic) is explained when factors A and B vary between low and high settings.
The team recognizes the advantage of using a pie chart to graphically represent the practical significance of the
effects exerted by the factors, and note that the remaining 10% represent the interaction effect (factors AB).
Finally, the Black Belt indicates that a Pareto Diagram can also be used instead of a Pie Chart.
The Black Belt recalls that before considering the practical significance we must evaluate the statistical
significance of the different effects.
Plotting the Main Effects (2 2)
_ _ n=Number of
EffectA = ∆y = y A + − yA − replicates taken at
or the treatment
Effect A = 1 {contrast A }
2n combination
Y Main Effect Plot for A

10.5
Average Y at
High condition
of factor A
3.5
Analysis of Factor A
Average Y at
Low condition
Y Main Effect Plot for B
Y
of factor A 10.0
B:+1
15.0 -1 +1 Average Y at
Factor A High condition
of factor B
10.5
∆Y 4.0
B:+1
6.0 Average Y at
B:-1 Low condition
Improve
5.0
3.5 of factor B
1_05_01_034
2.0 B:-1
-1 +1
A:-1 A:+1 Factor B
Plotting the Main Effects (22)
After conducting an experiment and recording the responses, the team asks the Black Belt to explain the next
steps.
We perform an analytical procedure by calculating and plotting the Main Effects. In our experiments we can see
that when factor A is high (+1), the average output equals 10.5, and when it's low (-1), the average equals only
3.5. The change in the output i.e. 7.0 can only be attributed to the difference between the levels of factor A. This
is because the settings neutralize both the effects due to factor B, and the interaction AB by the combination of
their – and + at each level of A. This is the result of contrasts that are balanced and orthogonal in well-designed
experiments.
The Black Belt then asks "Which settings maximize output, and what will be the output level for such settings?" A
team member answers "When factor A is set at high, the output average is 10.5".
"You are partially correct", says the Black Belt, "…however, we cannot only consider factor A. We must also
consider factor B. Another team member adds "Maximum output is 15.0 when factor A is high and factor B is
also high. "This is the correct and complete answer" says the Black Belt.
The team calculates the effect of factor A and then they conclude that factors A and B have the different effects.
Finally, they acknowledge the advantage of plotting the main effects to depict the results of the experiments, and
to visualize the difference in the response output between low and high settings for each factor.
Plotting the Interaction Effect (2 2)
_ _
n=Number of
Effect AB = ∆y = y AB + − y AB −
replicates taken
or
= 1 { } at the treatment
Effect AB contrast AB combination
2n
Interaction Plot for AB

Y
15.0
Both lines are
not parallel B:+1
Analysis of Interaction AB
thus indicating an Y
interaction effect 15.0
B:+1
6.0
5.0 B:-1 8.5

B:+1 ∆Y 6.0
5.5
B:-1
Improve
5.0
B:+1
1_05_01_035
2.0
B:-1 2.0 B:-1
A:-1 A:+1 A:-1 A:+1
Plotting the Interaction Effect (22)
"During the analytical procedure of the experiment, we extract relevant information" explains the Black Belt, "A
major part of this is to determine if there is an interaction effect between factors".
"Interaction occurs when the effect of a factor is a function of the level of another factor. For example, when
factor A changes from its low to high level, and factor B is kept at its low level, output changes by an average of
4.0 units (from 2.0 to 6.0). When B is kept at its high level and factor A varies from its low to high level, the
output changes by an average of 10.0 units (from 5.0 to 15.0). This is an indication of interaction between factors
A and B, i.e. they have a combine effect on the output."
"If the lines in the interaction plot are parallel, then there is no interaction between the factors. Conversely, if the
lines are not parallel, an interaction exists between the factors. Note that graphs should be used to visualize the
result of the ANOVA table. Graphs are useful to visualize significant interactions between factors but they should
not be the only means of interpreting data."
"Do interactions apply to transactional processes"? ask a team member. "Yes", replies the Black Belt, "I
remember a project where we analyzed invoices that were overdue. The first factor was "Customer", and two
levels were considered: "Airlines" and "Manufacturers". The second factor was "Invoice value". We were
surprised to discover that there was a important interaction effect between the two factors (Invoice value and
Customer). The Black Belt adds: "two factor interactions are common. However, three factor and higher are
rarely seen in practice in our type of industry.
Statistical Significance and
Practical Significance (2 2)
Output
Standard
Run Order A B AB Y1 Y2 Y3
1 (1) -1 -1 1 40.7 38.6 38.4
2 a 1 -1 -1 74.2 68.4 64.4
3 b -1 1 -1 44.1 42.1 41.1
ca lly
4 ab 1 1 1 72.6 86.0 85.6
ifi ca
nt
Residual
gn ti
Si atis
Degree Of Sums-of- Mean Error
St
A*B
Source Freedom Squares Squares F-value p-Value Significant
4%
A 1 3,543.20 3,543.20 164.12 0.0000 *** 2%
B 1 182.52 182.52 8.45 0.0197 *** B
A*B 1 63.48 63.48 2.94 0.1247
Residual Error 8 172.71 21.59 5%
Total 11 3,961.92
Sums-of- Percentage
Source Squares % Practically
A 3,543.2 89%
B 182.5 5% Significant A
Improve
A*B 63.5 2%
1_05_01_036
Residual Error 172.7 4% 89%

Total 3,961.9 100%
Statistical Significance and Practical Significance (22)
An engineer consults with a Black Belt after conducting a two-level, two factor experiment (factors A and B).
Three replicates per setting were done, and now he evaluates the statistical significance of the main effects for
each factor and the effect of interaction between factors A and B.
The Black Belt states that the engineer correctly conducted a two way ANOVA using Minitab. "The F test
indicates p values for factors A and B are smaller than 5%. This means that changes in factors A and B
significantly affect the output of the process. It is a clear indication that the change is not due to chance, and that
the condition can be reproduced. It is what we call Statistically Significant".
The Black Belt also highlights the importance of verifying the practical significance of the results using the Sums
of Squares. "This is called Practical Significance and in this case changes in factor A represent 89.4% of the
total variation."
The engineer recognizes that although factor B is Statistically Significant it does not account for more than 5 %
in practical terms, thus with little interest for him at this stage.
Residual Analysis (2 2)
Normal Plot shows Shows no residuals out of

no non-normality control beyond the control limits
Residual Model Diagnostics

Normal Plot of Residuals I Chart of Residuals
3 5
4 3.0SL=3.909
2 3
1 2
Residual
Residual
1
0 0 X=0.000
-1 -1
-2
-2 -3
-3 -4 -3.0SL=-3.909
-5
-2 -1 0 1 2 0 5 10
Normal Score Observation Number

4 3
2
3
1
Frequency
Residual
2 0
-1
1
-2
0 -3
-3 -2 -1 0 1 2 40 50 60 70 80 90
Residual Fit
Improve
1_05_01_037
Histogram of Residuals Variable does not change

normally distributed around 0 across values of the Output
Residual Analysis (22)
While analyzing his first 2 x 2 factorial design, a Black Belt notices that in Minitab he can graph the residuals. To
refresh the concept of residuals, the Black Belt consults with a Master Black Belt.
"When conducting a DOE, our objective is to test the appropriate hypotheses regarding the treatment of effects,
and to estimate them. The models of the errors are assumed to be independent random variables that are
2
normally distributed with mean equal to zero and variance ? , which is assumed constant for all levels of the
factors."
The Black Belt notices: "First, we must verify that residuals are normally distributed. The normal plot shows dots
aligned on a straight line indicating that residuals are indeed normally distributed."
"Second, we verify that residuals are normally distributed center around zero. The histogram of the residuals
shows this normal distribution and the centering around zero."
"Third, we consider the Individual Chart of Residuals. The dots are randomly distributed, no data out of the
control limits and no particular pattern, thus we may conclude that the residuals are randomly distributed
regarding the experiments run order."
"Finally, we check the random distribution of the residuals versus the settings. The fourth graph shows no
particular pattern and the dots are randomly distributed, thus we conclude that the residuals are normally
distributed."
The Black Belt remembers the importance to analyze residuals prior to any conclusions about experiment result.
Modeling - 2 2
Estimated Effects and Coefficients for yield

Term Effect Coef StDevCoef T P
Constant 58.017 1.341 43.25 0.000
A 34.367 17.183 1.341 12.81 0.000
B 7.800 3.900 1.341 2.91 0.020
A*B 4.600 2.300 1.341 1.71 0.125
Analysis of Variance for yield

Source DF Seq SS Adj SS Adj MS F P
Main Effects 2 3725.72 3725.72 1862.86 86.29 0.000
2-Way Interactions 1 63.48 63.48 63.48 2.94 0.125
Residual Error 8 172.71 172.71 21.59
Pure Error 8 172.71 172.71 21.59
Total 11 3961.92
ˆ =58.017 +17.183 * A +3.9 * B

Y
Improve
1_05_01_038
Modeling – 22
"The results of the DOE quantify the effect that the two factors and the interaction have on the process
response" says the Champion. He then continues, "are we limited two these factor settings, or is it possible to
know more about the process behavior?"
"It is correct to say that the DOE analysis gives us a good knowledge about the factor effects and their
interaction" answers the Black Belt. "However, there are other things to understand. From the mathematical
model we can see that if no change is made, then our process will have an average yield of 58.01% . We can
also see that the level of Factor A (Temperature) is key in controlling the yield of the process as compared to
Factor B (Pressure)."
"The average response output Yield when Temperature (A) and Pressure (B) are set to their High level is
computed using the model: Y = 58.017 + 17.183*1 + 3.9*1 = 79.1. With a similar calculation, we can determine
that when A is High and B is Low, the average yield will be Y = 58.017 + 17.183*1 - 3.9*(-1) = 71.3."
"We usually assume a linear response, in such a case this mathematical model is valid", says the Black Belt. "To
confirm this linearity, we use other techniques such as center points or multi-level factorial design"
The Champion recognizes the advantage to represent the process behavior using a mathematical model which
makes it possible to estimate process output for factor levels other than the pre-established settings.
Multi-Factor Experiment
On Up
Lo Hi
Off In Out Front Back Left Right Down
2 × 2 × 2 × 2 × 2 × 2 × 2
5 factors : 2 5 = 32 combinations
Improve
1_05_01_039

Multi-Factor Experiment
A Black Belt from a manufacturing area is discussing her project with a Master Black Belt. "During the team's
brainstorming session, we have identified several factors that can influence our CTQ characteristic, the
angularity of the parts formed on the Fluid Cell Press. We would like to do an experiment with at least 5 factors,
but this would be too large an experiment to do a full factorial. We had identified these factors a long time ago
but we don't know yet which ones have the biggest impact, "leverage variables". So far, I have calculated that
with these 5 factors at two levels, I will need 32 runs. With 6 factors, 64 runs and, 7 factors will require 128 runs -
and I have not included any replications yet. What do you recommend I do?"
The Master Black Belt replies, "if there are many factors in the experiment, a full factorial experiment will involve
a large number of tests, even when only two levels of each factor are being investigated. So, when you don't
know which factor has a substantial effect on a response variable, and you want to verify several factors, then
the fractional factorial design is the preferred analytical tool. In fact, it is common practice to do a fractional
factorial experiment when we have 5 factors or more."
Fractional Factorial Designs
Full Factorial Fractional Factorial
Example with 4 Factors

Main Interactions Main Interactions
Effects Effects
A AB, AC, AD, BC, BD, CD A AB, AC, AD, BC, BD, CD
B
Improve
B ABC, ABD, ACD, BCD
1_05_01_040
C
C ABCD
D
D
Fractional Factorial Designs
A Black Belt is presenting her Design of Experiment plan to a Champion: "For my project on the Fluid Cell Press,
I will use a fractional factorial design. We want to study four factors at two levels. Instead of doing a full factorial
4
design, which represents 16 runs (2 ) and is very time consuming, we will do a half fraction of it, which consists
of 8 runs."
"I don't understand how you can do that. How will you be able to test all the possible factor combinations if you
do not use a full factorial experiment?", asks the Champion.
"A fractional factorial design is a carefully prescribed subset of all possible combinations of a full factorial design.
Statisticians have shown that for 5 factors or more, you can obtain nearly the same information as a full factorial
with a well-designed fractional factorial experiment. When compared to a full factorial design, a certain amount of
information will be lost, but the impact is minimal. The trade-off is mainly that we lose information regarding the
interactions between three factors or more. However, for most situations in industry, it is considered reasonable
to assume that third order and higher order interactions (interaction between three factors or more) are
statistically negligible."
"The benefits of conducting a fractional factorial experiment are that fewer tests are required compared to the full
factorial design, reducing the required time and resources, and yet we can still assess many factors at the same
time."
Forming a Half-Fraction of a Full
Three Factor Design
Run A B C AB AC BC ABC
1 -1 -1 -1 1 1 1 -1
2 1 -1 -1 -1 -1 1 1
3 -1 1 -1 -1 1 -1 1
4 1 1 -1 1 -1 -1 -1 Generator: ABC=1
5 -1 -1 1 1 -1 -1 1
6 1 -1 1 -1 1 -1 -1
7 -1 1 1 -1 -1 1 -1
8 1 1 1 1 1 1 1
Same
Contrast
Run A B C AB AC BC ABC
Confounding : A & BC 2 1 -1 -1 -1 -1 1 1
or alias B & AC 3 -1 1 -1 -1 1 -1 1
Improve
1_05_01_041
structure C & AB 5 -1 -1 1 1 -1 -1 1
8 1 1 1 1 1 1 1
Forming a Half Fraction of a Full Three Factor Design
"I know that Minitab creates fractional factorial designs, but for my own understanding I am interested in knowing
how to construct one. Could you explain this to me?", asks a Black Belt to a Six Sigma Instructor.
"OK, we will take a simple example with a three factor design. For most situations in our industry, it is considered
reasonable to assume third order interactions to be statistically negligible, so we will use the ABC interaction to
create the fraction. First, we have to sort the entire matrix by the highest order interaction. In this instance, we
must sort on the ABC column. The fractional design is formed by selecting all those treatment combinations
where the ABC effect is either "1" or "-1"."
"The sign of the ABC effect constitutes a "block". Therefore, the design generator can be ABC=-1 or ABC=1. In
our case, if we choose the generator ABC=1, we will keep only the runs 2, 3, 5 and 8. Look what happens to the
matrix. First, the ABC effect is lost since it was used as the generator to form the half fraction of the full factorial
matrix. Second, the unique effect of "A" can not be separated form the unique effect of the BC interaction
because both columns have the same sign at each experimental run. The same thing happens for B with AC,
and C with AB. This is called confounding or alias structure. Note that I could have chosen ABC=-1 as the
design generator and arrived at the same conclusions."
"The design becomes these 4 runs and we only need the three columns for A, B, and C. The other columns
constitute a repetition of these three. For three factors with this design, the main effects are confounded with the
two factor interactions."
Design Resolution
Resolution Main Confounding Pattern Use

III Main effects are confounded To screen many factors
with second order interactions (Screening Designs)
IV Second order interactions are To study uncomplicated
confounded with other second effects and interactions
order effects interactions (Characterization Designs)
V Second order interactions are Same as resolution IV.
confounded with third order For resolution V and higher:
interactions to study more complicated
effects involving 2 or 3fac-
tors and obtain amathema-
Improve
tical model.
1_05_01_042
(Optimization Designs)
Design Resolution
"In Minitab, if I use the function "Create Factorial Design" (under Stat > DOE), I have noticed that for a specific
number of factors that we want to study, different types of designs are available. I am not sure that I understand
the terminology associated to the designs, could you explain this to me?", asks a Black Belt to a Master Black
Belt.
"Yes, sure! Depending on the number of factors and runs, the resolution defines the alias or confounding
structure of a design. For example, a resolution III design means that the main effects are confounded with
interactions between factors." "That design doesn't seem to be very useful", says the Black Belt. "Well, these
designs are called Screening Designs and a small number of runs are required to use them", replies the Master
Black Belt. "However, the quantity of information is limited, so these designs are generally used to initially screen
a large number of factors, as their name implies. Another experiment is then needed to study, in more detail, the
important factors that have been found with the resolution III design and their interactions effects."
"With resolution IV and V designs, main effects are not confounded with interactions but interactions are
confounded together", continues the Master Black Belt. "We consider them as Characterization Designs and
they are generally used to study relatively uncomplicated effects and interactions from a smaller number of
factors compared to resolution III designs. Resolution v and higher designs are called Optimization Designs and
are most often used to study more complicated effects and relationships involving two or three factors. Such
designs are also employed to obtain a mathematical model describing the relationship between the CT
characteristic and the factors (Y = f(X1, …, XN)).
Choosing a Factorial Design
Number Number of factors

of Runs*
2 3 4 5 6 7 8
4 full III
8 full IV III III III
16 full V IV IV IV
32 full VI IV IV
6 −2
2
64 full VII V
128 full VIII
IV
Improve
1_05_01_043
*Note: the number of runs is for 1 replication.
Choosing a Factorial Design
"For my project on the automatic riveting machine, six factors have been selected as potential Critical to Process
characteristics. For each of them, we have identified two levels (high and low) that can be used in an
experiment. What design do you suggest I use?" asks a Black Belt to a Master Black Belt.
"For six factors, there are four types of factorial designs that are available: a resolution III design which consist of
8 runs, a resolution IV design with 16 runs, a resolution VI with 32 runs and a full factorial with 64 runs. The
design you select will depend on what you need to know, your level of knowledge of the process and resource
availability." The Black Belt replies, "well, it is my first experiment on this machine. From the Analysis Phase
certain factors appeared to have an influence on our CTQ, but we don't really know much about them.
Interactions between factors are possible but we don't know if they exist."
"In this situation, you don't have a lot of knowledge about the factors and you are definitely not at an optimization
stage, I suggest you use the resolution IV design with 16 runs, replies the Master Black Belt. Don't forget that
you will have to include a few replications which will significantly increase the amount of runs."
6-2
"Why is this design called a 2 ?" asks the Black Belt. "This refers to the notation that is used to describe
k-p
factorial design. In general we use 2 where "2" indicates that the factors are studied at two levels, "k"
represents the number of factors and "p" what fraction of a full factorial the design is. Then, p=1 it means a ½
fraction of the full factorial, p=2 means a ¼ fraction and so on. In your case, p=2 means that your design is ¼
fraction of a full factorial for 6 factors at two levels."
Objective, Response Variables
and Factors
CTQ
target:
90 O
CTQ: Angularity
Machine Tools Materials
Angularity of
the parts
Improve
1_05_01_044
People Methods Environment
Objective, Response Variables And Factors
A Black Belt presents her Design of Experiment plan to a group of Champions. "The objective of the Design of
Experiment is to identify the factors that affect the quality of the parts formed in the Fluid Cell Press. The CTQ
characteristic that we want to improve is the angularity of the parts. This CTQ will be measured on two typical
parts that we produce: "curved" and "straight" parts. Two separate statistical analyses will be done, one for each
type of part."
"We regularly rework the parts produced in this process. Can you please clarify the problem with the CTQ?",
asks a Champion. "Sure", replies the Black Belt. "The parts produced have an angle that do not meet the target
of 90 degrees. An operation of check and straighten has to be done practically on every part to bring the CTQ on
target."
"Do you have a good measuring system to obtain accurate data for your DOE?", asks the Champion from the
Quality department. "Yes, we have organized a location where all the parts will be measured, and a Gage R&R
study has been conducted with the team members who will measure the parts. The result of the Gage R&R
study detected a problem with the measuring system, but it has been corrected."
"The Black Belt continues, "with the information gathered since the beginning of the project and a recent
brainstorming session, we have identified more than 60 causes that can affect the angularity of the parts. We
used a fishbone diagram to summarize the causes in the following categories: Machine, Tools, Materials,
People, Methods, and Environment."
Select Factors and Levels
Factors Levels
Low (-1) High (1)
A: Loading of tray 20% 50% In general, the levels
B: Height of blocks 1/4” 1 1/4” should be set wide but
C: Grain direction 45º 90º not so much that they
extend beyond the
D: Covering type Soft Hard
operational range of
E: Pressure 8,000 PSI 15,000 PSI the factor.
F: Forming Speed Normal Slow
G: Spring-back 2º 4º
Y
True Effect
Response
Experimental Effect
Improve
1_05_01_045
Lo Hi
Factor Settings
(-1) (+1)
Select Factors And Levels
During a presentation of a DOE to a group of Champions, a Black Belt explains the selection of factors and
levels. "With the information we gathered since the beginning of the project on the Fluid Cell Press, and a recent
brainstorming session, we have identified seven factors that can influence the CTQ characteristic (Angularity of
the parts)."
"This process is so complex. How did you select these seven factors? I am sure that there are more than seven
factors", says the Methods' Champion. "Initially, more than 60 causes were identified but we classified these
causes to narrow down the number of factors to study in the experiment", replies the Black Belt. First, we
separated the causes into three different categories: controllable, noise and held constant. The controllable
factors are studied in the experiment because we can control and change them during an experiment. From the
group of the controllable factors, we used a Nominal Group Technique where each team member selected and
rank the factors of his choice by order of importance. Finally, the 7 factors were identified."
"Is it a reasonable number of factors for an experiment?", asks a Champion. "Well, generally the number of
factors studied in an experiment is less than 15, replies the Black Belt. However most industrial experiments are
done with 8 factors or less."
"How did you select the levels?", asks another Champion. "Some factors such as forming speed only have two
options", replies the Black Belt. Other levels come from the operators' experience and by pushing the factors to
reasonable extremes. In general, the levels should be set "wide", but not so much that they extend beyond the
operational range of the factor to have a better chance of detecting the effects that are present."
Alias Structure
Alias Structure for a 2 7-3
iv design
A + B*D*G + B*E*F + C*D*F + C*E*G

B + A*D*G + A*E*F + C*D*E + C*F*G
C + A*D*F + A*E*G + B*D*E + B*F*G
D + A*B*G + A*C*F + B*C*E + E*F*G
E + A*B*F + A*C*G + B*C*D + D*F*G
F + A*B*E + A*C*D + B*C*G + D*E*G
G + A*B*D + A*C*E + B*C*F + D*E*F
A*B + D*G + E*F
A*C + D*F + E*G
A*D + B*G + C*F
A*E + B*F + C*G
A*F + B*E + C*D
A*G + B*D + C*E
B*C + D*E + F*G
Improve
A*B*C + A*D*E + A*F*G + B*D*F + B*E*G + C*D*G + C*E*F
1_05_01_046
Note: The terms presented here are only up to the third order interactions
Alias Structure
"I have the results of my DOE on the Fluid Cell Press, but I am not sure which function to use in Minitab to
analyze them. Can you please help me?", asks a Black Belt. "Sure, what type of experiment did you conduct?",
asks the Master Black Belt. "A fractional factorial experiment with seven factors at two levels each. I used a
7-3
resolution IV design, 2 with 2 replications, so a total of 32 runs were done."
"Since it is a resolution IV, we know that no main effects are confounded with any other main effect or second
order interaction. But second order interactions are confounded with other second order interactions. First, we
verify the alias or confounding structure using the DOE function of Minitab."
"This function confirms that we have a resolution IV design. The main effects are confounded with third order
interactions, and second order interactions are confounded with other second order interactions. For most
situations in industry, it is considered valid to assume that third order and higher interactions are negligible. To
perform a statistical analysis such as ANOVA, we can input in Minitab the following terms for the factors of the
model: A, B, C, D, E, F, G, A*B, A*C, A*D, A*E, A*F, A*G and B*C."
"Note that since the interactions are confounded with other interactions, we cannot separate the effect. For
example, if the A*B effect is significant, it can be either A*B, D*G or E*F that is important, because in the design
matrix, the same contrast is used for these three interactions. However, we use the first interaction as a label
because when we analyze the data, we have to consider that the label A*B represents in fact A*B+D*G+E*F."
ANOVA Table
Analysis of Variance for Angularity

Source DF Seq SS Adj SS Adj MS F P
A 1 9.753 9.753 9.753 41.06 0.000

B 1 6.722 6.722 6.722 28.30 0.000
C 1 10.889 10.889 10.889 45.84 0.000
D 1 375.836 375.836 375.836 1582.26 0.000
E 1 2.170 2.170 2.170 9.14 0.008
F 1 1.389 1.389 1.389 5.85 0.027
G 1 583.679 583.679 583.679 2457.27 0.000
A*B 1 134.070 134.070 134.070 564.43 0.000
A*C 1 13.133 13.133 13.133 55.29 0.000
A*D 1 3.445 3.445 3.445 14.50 0.001
A*E 1 3.897 3.897 3.897 16.40 0.001
A*F 1 5.147 5.147 5.147 21.67 0.000
A*G 1 3.230 3.230 3.230 13.60 0.002
B*C 1 1.188 1.188 1.188 5.00 0.039
Improve
Error 17 4.038 4.038 0.238
1_05_01_047
Total 31 1158.586
ANOVA Table
A Master Black Belt explains to a Black Belt how to analyze the results of a DOE in Minitab. "In order to obtain
the sums-of-squares of each term in a model, the function "General Linear Model" can be used".
"Is it really important to calculate all the sums-of-squares?", asks the Black Belt. "Yes", replies the Master Black
Belt. "The sums-of-squares are the basic measure of variation associated with each term of the model. They are
also used to evaluate the practical significance of the effects."
7-3
"My design is a fractional factorial 2 of resolution IV. How do I know which model to use in Minitab?, asks the
Black Belt. "Generally, for a resolution IV design from the alias structure, we select the main effect and the two
factors interaction", replies the Master Black Belt.
"The ANOVA table shows that all the effects are statistically significant at an alpha level of 0.05. Although they
are all statistically significant, they are not all practically significant as demonstrated when we consider the
magnitude of each sum-of-squares compared to the total sum-of-squares. It seems that D, G and A*B have the
largest sums-of-squares compared to the total."
"Don't forget that A*B is only a label here. If we look at the Alias structure associated with your design, we see
that the A*B interaction is confounded with D*G and E*F. Since D and G are both important as main effects, it is
more likely that the D*G interaction is present here. The next step is to graph the residuals to verify the
assumptions associated with ANOVA, and to detect the presence of outliers that can contaminate the results."
Practical Significance
Pareto Chart for Source
1200
100
1000
80
800
Percent
Count
60
600
40
400
200 20
0 0
ers
Source G D A*B A*C Oth
Count 583.679 375.836 134.070 13.133 51.868
Percent 50.4 32.4 11.6 1.1 4.5
Improve
Cum % 50.4 82.8 94.4 95.5 100.0
1_05_01_048
"How can I identify the effects that are really important? The ANOVA table shows that almost all the effects are
statistically significant", asks a Black Belt. "This situation happens especially when the experimental error is
small", replies the Master Black Belt. To estimate the practical significance, we use the sums-of-squares from an
ANOVA table to create a Pareto or a Pie chart."
"From the Pareto chart we see that the main effect G represents 50.4% of the total variation of the response
variable (CT characteristic), and that D represents 32.4 %. It seems that G and D are the vital few variables
since they account for more than 80 % of the variation in the response variable."
"The A*B effect represents 11.6 % of the total variation. What does it mean taking into account the fact that the
7-3
design is a fractional factorial 2 of resolution IV?", asks the Black Belt. "From the alias structure associated
with the design, we know that other interactions are confounded with the A*B interaction. In this case, D*G and
E*F are confounded with A*B, and since both D and G are important main effects, there is a good chance that
the interaction D*G is exerting the most influence here."
Normal Probability Plot of the
Effects
Normal Probability Plot of the Standardized Effects

(response is Angulari, Alpha = .10)
A: A
G B: B
C: C
AC D: D
1 E: E
C F: F
AE G: G
Normal Score
AD
E
F
0 ABC
BC
AG
AF
B
-1 A
AB
-40 -30 -20 -10 0 10 20 30 40 50
Improve
1_05_01_049
Standardized Effect
Normal Probability Plot of the Effects
"A graph used to analyze the results of a DOE is the Normal Probability Plot of the effects, says a Master Black
Belt to a group of Black Belts. "What is the difference between this graph and a Pareto chart of the sums-of-
squares?", asks a Black Belt. "The main difference is that instead of using the sums-of-squares, the Normal
Probability Plot represents the estimated contrast effects but both graphs are used to identify the important
effects", replies the Master Black Belt.
"How do I interpret this plot?", asks a Black Belt. "Well, If all of the contrasts represent nothing but random
variation (i.e. if there are no real differences between the averages at -1 compared to 1), then the contrast
effects should form a reasonably straight line on the Normal Probability Plot. However, if some contrasts
represent real differences rather than random variation, then these contrasts are plotted on the lower left or
upper right of the straight line formed by the other contrasts. In other words, the "outliers" represent significant
effects."
"We can clearly see in our plot that G, D and A*B represent real differences. Therefore, G and D are the vital few
variables, and A*B is a significant interaction effect. From the alias structure associated with our design, we
know that D*G and E*F are confounded with A*B, and since both D and G are important main effects there is a
good chance that the interaction D*G is exerting the most influence here."
Main Effects Plot
Main Effects for Angularity
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
14
Angularity
12
10
6
A B C D E F G
FACTORS
Improve
1_05_01_050
Main Effects Plot
"I find that the ANOVA table is not user friendly because it displays many numbers arranged in a tabular form. Is
there a tool that creates a visual representation of the results?", asks a Black Belt. "The Main Effects Plot is a
very useful tool", replies the Master Black Belt. "The mean of the response variable is plotted for the levels of
each factor. It helps us to visualize and confirm the important main effects."
"The Main Effects Plot shows a large difference in the mean of the response variable between the low (-1) and
high (1) levels of factors D and G", says the Master Black Belt. "Those are the exact same results I found using
the sums-of-squares in the ANOVA table", says the Black Belt. "
"Where does this data come from?", asks the Master Black Belt. "The data is the result of a fractional factorial
design related to our project", replies the Black Belt.
"What is the CT characteristic and the type of design you used?", asks the Master Black Belt. "The CT
7-3
characteristic is the angularity of the parts, and the design is a fractional factorial 2 of resolution IV with two
replications. Since our objective is to minimize the response variable (CT characteristic), we should set factor D
at its high level (1) and factor G at its low level (-1)", replies the Black Belt. "If there is an interaction between
these two factors, it is preferable to do an interaction plot to find the best setting for both factors", says the
Master Black Belt.
Interaction Plot
Interaction Plot for Angularity
D
20 -1
1
-1
1
15 D Low (-1)
Mean
10
D High (1)
Improve
-1 1
1_05_01_051
Interaction Plot
"When do we use the Interaction Plot in a DOE?", asks a Black Belt. "We use it when the results of an ANOVA
table make us suspect the presence of an interaction. The Interaction Plot helps us to visualize and confirm this
interaction", replies the Master Black Belt.
"Since I believe that there can be an interaction between factors D and G in our DOE, I create an Interaction
Plot. Since the lines are not parallel, we confirm that there is an interaction between factors D and G."
"To interpret this plot we must consider our objective. Do you need to maximize or minimize the response
variable?", asks the Master Black Belt. "I need to minimize it", replies the Black Belt. "We then can see that when
factor G is set at its low level (-1), the response variable is smaller compared to when G is set at its high level
(1). We can also see that when factor D is set at its high level (1), the difference in the response variable
between "G low" and "G high" is smaller than when D is set at the low level (-1). When combined with factor G,
factor D has a large impact on the mean of the response variable."
"Considering that we want to minimize the response variable, we conclude from this graph that factor D must be
set at its high (1) level, and factor G at its low (-1) level."
Blocking Variable
Non-homogeneous units Formed into blocks
Batch 1 Batch 2
StdOrder RunOrder Block A B C

7 1 1 -1 -1 1
6 2 1 -1 1 -1
8 3 1 1 1 1
5 4 1 1 -1 -1
3 5 2 1 -1 1
Improve
4 6 2 -1 1 1
1_05_01_052
1 7 2 -1 -1 -1
2 8 2 1 1 -1
Blocking Variable
A Black Belt planning a DOE related to a process in the composites material department says: "I am afraid that
by using different batches of material during our DOE, the results will be "contaminated". After all, every batch of
material is different from the others". A Master Black Belt replies, "to reduce the effect of noise variables such as
batches of material, shifts, operators, suppliers, etc., we use "Blocking Variables".
"When using Blocking Variables, the experimental runs are sub-divided into groups in such a way that the runs
within a group are relatively homogeneous. The reason for grouping units or runs into blocks is to reduce the
amount of experimental error or noise, since it makes it difficult to identify factor effects. However, when creating
a design matrix, we must not forget that generally a Blocking variable is confounded with higher order
interactions".
"To remove the effect of the added noise, we use a "randomized block design" where the homogeneous units
are formed into blocks, and the run within each block is performed in a random order."
"Blocking variables allow us to remove the noise from the error term by creating a separate term. Thus, blocking
provides two benefits: First, it reduces the experimental error which makes it easier to detect effects; and
second, the experimenter can test the significance of the blocking variable."
Analyze an Experiment With a
Blocking Variable
Y: Filtration Rate A
Blocking Variable: Days 1 AD
Normal Score
A: Temperature D
C
B: Pressure 0
C: Fomaldehyde Concentration
D: Stirring Rate
-1
Analysis of Variance for Filtrate
AC
Source DF SS MS F P
Block 1 7.56 7.56 0.25 0.642 -5 0 5
A 1 1870.56 1870.56 62.22 0.001 Standardized Effect
B 1 39.06 39.06 1.30 0.318
C 1 390.06 390.06 12.98 0.023
D 1 855.56 855.56 28.46 0.006
A*B 1 0.06 0.06 0.00 0.966
A*C 1 1314.06 1314.06 43.71 0.003
A*D 1 1105.56 1105.56 36.78 0.004
B*C 1 22.56 22.56 0.75 0.435
Improve
B*D 1 0.56 0.56 0.02 0.898
1_05_01_053
C*D 1 5.06 5.06 0.17 0.703

Error 4 120.25 30.06
Total 15 5730.94
Analyze an experiment with a Blocking Variable
During a coaching session, a Six Sigma Black Belt says, "When conducting a DOE on a chemical process, I
followed your recommendation and used a blocking variable. Since it took two days to complete the experiment,
the variable "days" was the blocking variable. However, I don't know how to analyze the data. Two days can
have a large effect on any chemical process and I don't want to increase the amount of experimental error, thus
making it more difficult to identify the factor effects."
The Master Black Belt replies, "Well, you just have to put the blocking as a term in the model used, and analyze
it with the other factors contained in a detailed ANOVA table".
"If the block effect is not statistically significant, we can continue the analysis without considering the blocking
variable, and if necessary, redo the model without considering the blocking variable. If the block effect is
significant, then we can proceed with the analysis, knowing that the results are not contaminated. Statistical
significance means that we took an accurate decision in using such blocking variable."
"From the ANOVA table we see that, for an alpha level of 0.05, the block effect is not statistically significant, and
that factors A, C and D, as well as the interaction effects A*C and A*D are statistically significant. The practical
significance of these factors and interactions is confirmed when we compare their sums-of-squares with the total
sums-of-squares. The same conclusion is obtained when we use the Normal Probability Plot of effects.
After completing these steps, we continue our analysis with the prescribed steps (Residual analysis, main effects
plots, and interaction plots)."
Two-Factor Multi-Level
Full Factorial
CTQ Y: Composite part
(Measurement Scale: 0 - 100 where 100 is perfect part) Minitab Entry
Shown in Standard Order
Factor A: Curing Pressure
A B Y
Factor B: Curing Temperature 1 1 40
1 2 50
Data Collection Sheet 1 3 60
1 1 20
A B Y1 Y2 1 2 30
1 3 50
1 1 40 20
2 1 50
1 2 50 30 2 2 70
1 3 60 50
2 3 50
2 1 10
2 1 50 10
2 2 60
2 2 70 60 2 3 60
3 1 30
2 3 50 60
3 2 20
3 1 30 40 3 3 70
3 1 40
3 2 20 10
3 2 10
Improve
1_05_01_054
3 3 70 80 3 3 80
. . . So what calculations must be made ?

Multi-Level Two-Factor Design
The Champion responsible for the manufacturing of composite parts believes that the autoclave settings are not
optimal. Current settings create problems related to shape and other CTQ characteristics of a particular group of
parts.
From previous screening studies, the team assigned to this project identifies two variables which influence the
process:
• Curing Pressure (factor A);

• Curing Temperature (factor B).
The Six-Sigma team adopts a measurement scale for the output (Y) that ranges from 0 to 100, where 100 is a
perfect composite part. The team prepares to conduct a two-factor three-level DOE with two replicates.
To collect the data in an organized way, the Black Belt creates this data table in Minitab showing the 18 runs. To
code each setting for factors A and B, she chooses 1 as the label to denote a low setting, 2 for an intermediate
setting and 3 for a high setting.
After conducting the experiment and recording the responses, the team is ready to analyze the data using
Minitab's statistical functions.
Statistically Significant?
Multi-Level Full Factorial
A B Y
1 1 40
1 2 50
1 3 60
1 1 20
1 2 30
1 3 50
2 1 50
2 2 70
2 3 50
2 1 10
2 2 60
2 3 60
3 1 30
3 2 20
3 3 70
3 1 40
3 2 10 The Anova Table
3 3 80
Improve
1_05_01_055
Statistically Significant
A Six-Sigma team conducts a DOE related to the curing process of composite parts. Two factors are considered:
Pressure (factor A); and Temperature (factor B). The team conducted two replicates for this two-factor three-
level experiment. After recording the responses, the Black Belt generates the ANOVA table in Minitab to
determine if there is a statistical difference between the output at different levels of the factors (Pressure and
Temperature).
"There are various ANOVA functions in the Minitab menu, which one do we choose?", asks a team member.
"In this case we choose "Balanced ANOVA", this allows us to see if the interaction between factors A and B has
an impact on the output", replies the Black Belt.
The Black Belt explains to the team the interpretation of the results shown in the ANOVA table.
"From this table we see that factor A (Pressure) is not statistically significant, for an ? risk of 5%. The p value is
0.466, meaning that there is a 46.6% probability that the change in the output is due to chance. On the other
hand, factor B (Temperature) has a great influence on the output. With a p value of 0.008, it is highly improbable
that the change in output is due to chance alone. The Interaction between factors A and B (A*B) has a significant
effect on the CTQ, as shown by a p value smaller than 0.05. We now can say that the change in the output is not
random in nature, and thus is repeatable."
Team members recognize that even if the main effect of factor A is not statistically significant, it affects the
output through its interaction with factor B.
Practically Significant ?
Analytical Significance
Analysis of Variance for Y
Source DF SS MS F P Significant
A 2 555.6 277.8 5.00 0.014 ***
B 2 5755.6 2877.8 51.80 0.000 ***
A*B 4 5577.8 1394.4 25.10 0.000 ***
Error 27 1500.0 55.6
Total 35 13388.9
Pie Chart of Source
Sums-of-Squares
B (5755.6 - 43%)
Source DF SS %
A 2 555.6 4.1
B 2 5755.6 43.0 A (55.6 - 4.1%)
A*B 4 5577.8 41.7
Error 27 1500.0 11.2
Error (1500 - 11.2%)
Total 35 13388.9 100.0
Improve
1_05_01_056
A*B (5577.8 - 41.7%)
Practically Significant
"There is something I do not understand," says the Champion, "You just told me that factor A (Type of coolant) is
statistically significant, but now you say that this factor is not significant. Can you clarify this apparent
contradiction."
The Black Belt replies "There are situations where certain factors are statistically significant, but not practically
significant. We conducted an experiment with two factors: A) Type of coolant and B) Type of end-mill. Each
factor was set at three levels. In the ANOVA table, we see that, for an ? risk of 0.05, the p value for factor A is
0.014, thus "type of coolant" is statistically significant. However, from the analysis of the Sum-of-Squares, we
see that this factor accounts for only 4.1% of the Total Sum-of-Squares (practical significance). In fact, we must
first take advantage of the effect caused by factor B, because this factor accounts for the largest percentage of
the total Sum-of-Squares (43.0 %)."
"Does this means that factor A is not to be considered?" asks the Champion.
"No" says the Black Belt, "We must not decide based only on main effects. As we can see, the interaction A*B is
statistically significant and also practically significant. In order to take advantage of this interaction, we must work
on both factors A and B."
The Champion recognizes that in order to maximize the process output Y, both factors, A and B must be
considered.
Residual Analysis
Residual Analysis for Composite Part
1 Normal Plot of Residuals 3 I Chart of Residuals
20 30 3.0SL=27.38
20
10
10
Residual
Residual
0 0 X=0.000
-10
-10
-20
-3.0SL=-27.38
-20 -30
-2 -1 0 1 2 0 10 20
Normal Score Observ ation Number
2 4
6 20
5
10
4
Frequency
Residual
3 0
2
-10
1
Improve
0 -20
1_05_01_057
-20 -15 -10 -5 0 5 10 15 20 10 20 30 40 50 60 70 80

Residual Fit
Residual Analysis
"We have conducted the DOE, interpreted the ANOVA table and identified the factors that have statistical and
practical significance. Does this mean that we have finished the DOE data analysis?" asks a team member.
"Not yet" replies the Black Belt, "There is a very important statistical analysis left to do. We must analyze the
residuals."
Using Minitab, the Black Belt performs the residual analysis and prints out the corresponding four graphs."
• The first shows that the residuals are normally distributed.

• The second is the histogram. It indicates that the residuals are well grouped around zero, as it should be if
nothing special is affecting the residuals.
• The third is the chart of the residuals function of sequence run order. We can see that the residuals are
randomly distributed and that no point lies outside of the control limits, which is what we expect to see in this
graph.
• Finally, the fourth graph reveals if the residuals are randomly distributed around the fitted values. The
residuals on this graph look alright.
The team members conclude the residuals are normally distributed around zero and that they are not affected by
special causes.
Main Effect Plot
A B Y
1 1 40
1 2 50 Main Effect plot - Means for Composite Part
1 3 60
1 1 20
1 2 30
1 3 50
2 1 50 63
2 2 70
2 3 50
2 1 10 55
2 2 60
2 3 60
47
Y
3 1 30
3 2 20
3 3 70
3 1 40 39
3 2 10
3 3 80
31
A B
Improve
The factor pressure (A) has relatively little effect on the response Y;
1_05_01_058
The factor Temperature (B) has a great effect on the output.
Main Effect Plot
An across functional team conducts a DOE during the design of an improved composite part. The team studies
the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine.
2
Each factor was set at three levels (1, 2 and 3) for a 3 experiment, and 2 replicates are conducted for each
setting of this full factorial design. The Black Belt plots the Main effects, and explains the interpretation to the
team.
"The graph confirms the results of the ANOVA table, and helps us to interpret the effects of the factors on the
output. Factor A (pressure) has relatively little effect since the average output remains almost constant at each of
its three levels."
"Factor B ( temperature) however, has a great effect on the output. We can see that average output is the
highest when factor B is set at level 3."
"Does this mean that in order to obtain the highest average output we should specify the autoclave's
temperature to the value corresponding to the level 3, and not worry about the
pressure?", asks a design engineer.
"It is too early to make that statement," replies the Black Belt. "We should study the interaction plot to find the
optimal setting for both factors."
Interaction Plot
A B Y
1 1 40 Interaction Plot - Means for Composite Part
1 2 50
1 3 60
1 1 20 A
1 2 30
1 3 50 75 1
2 1 50
2
2 2 70 65 A2 3
2 3 50 1
2 1 10 55 2
A1 3
2
2
2
3
60
60
Mean 45
3 1 30 Curvature on the CTQ
35 A3 output average of
3 2 20
3 3 70 Composite Part for B
3 1 40 25 (Temperature) average
3 2 10 when A (Pressure) is at
3 3 80 15 level 3
1 2 3
B
The largest output average correspond to factor A at level 3 and factor B at level 3
Improve
1_05_01_059
The smallest output average correspond to factor A at level 3 and factor B at level 2
Interaction Plot
An across functional team conducts a DOE during the design of an improved composite part. The team studies
the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine.
2
Each factor was set at three levels (1, 2 and 3) for A 3 experiment, and 2 replicates are conducted for each
setting of this full factorial design. After discussing the main effect plot, the Black Belt creates and interprets the
"Interaction Plot".
"We see three curves, each one corresponds to a particular level of factor A. The highest average output is
obtained when both factors A and B (pressure and temperature) are at their respective level 3. The lowest output
average is obtained when factor A (pressure) is set at level 3, and factor B (temperature) is set at level 2".
"When factor A is set at level 1, the output or response is almost linear for the different levels of factor B. Notice
2
that if we had conducted only a two-level design (i.e. 2 experiment), we would have missed the non linearity of
the response average corresponding to level 2 setting of factor B. That is, we would not be able to discover, and
take advantage, of the maximum or the minimum of the curves relative to factor A settings."
"Another interesting point to note is that the response average is independent of factor A when factor B is Low.
There is no output difference for A at level 1 and 2 when B is set at level 3 but a difference exists for factor A at
level 3 for this setting of B. The setting 2 for B is interesting since it gives the largest output difference between
the three settings of factor A."
The design engineers recognize that before specifying any setting, they should consider the Main Effect and
Interaction plots in order to identify the optimal settings.
Center points
Sometimes it is desirable to study quantitative factors at more than two levels. Examples of situations that would
lead to three or more levels include:
• Checking for quadratic effects (curvature);

• Estimating the error;
• Including the current level of the factors as well as the high and the low levels.
k
By adding center points to a 2 factorial design, we may address these concerns without a significant increase of
the number of experiments. "How do you add center point to your design?" asks the Champion.
"We perform "n" replicates run at the mid-level for each of the factors in the factorial design. In this case, the
ANOVA table shows that it exists a significant effect due to the curvature at the center point, since its p value is
less than 5%. The curvature cannot be assigned to a particular factor, but the center point gives an indication of
whether it is appropriate or not to interpolate between the factorial points."
"Note that the magnitude of the curvature effect can be directly compared to the factor effects for relative
importance if the number of center points is equal to one-half the number of factorial points. Also, the number of
center points does not affect the other effects."
The Champion recognizes that, with only 2 center points, the error is calculated and the curvature is detected,
2
needing only 6 experiments (i.e. 4 runs for the 2 with one replicate plus 2 runs at center point) compared to a
2
3 full factorial design which needs 9 runs.
Strategy of Good Experimentation

Step 1 – Define the Problem
Step 2 – Establish the Objective
Step 3 – Select the Response Variable(s)
Step 4 – Select the Independent Variable(s)
Step 5 – Choose the Variable Levels
Step 6 – Select the Experiment Design
Step 7 – Collect the Data
Step 8 – Analyze the Data
Step 9 – Draw Conclusions
Step 10 – Achieve the Objective
Strategy of Good Experimentation
"I want to conduct an experiment on my process", says an Black Belt to a Master Black Belt. "Can you outline
the steps I should take to ensure that it is successful?"
"We follow a 10 step process to ensure that we cover all the main topics of a good experiment. Steps 1 and 2
are to define the problem, and to establish the objective for the experiment. A well-constructed problem
statement, combined with a good objective statement for the planned experiment, is a key part of the
improvement cycle. The objective of the cycle, preferably with a statement of planned action, should be stated in
such a way that it provides guidance to those designing the experiment. The objective statement should clarify
whether the experiment involves a screening experiment, which is used to study a large number of variables, to
find which has the most important influence, studying a few variables in depth, or confirming the results of past
studies under new conditions.
"Where possible, a statement of the results required should be considered in the objective when appropriate. As
an example, the objective for an experiment might be to identify factors that call be used to improve yield by 5%,
or identify the factors that have the greatest effect on the variation of the parts."
Another consideration in the statement of the objective, is that it should be helpful in identifying the response
variables, and appropriate factors for the study", replies the Master Black Belt.
Team Selection for DOE’s
Key tasks for the Experimental Team:

• Write Problem Statement
• Define the Objective of the DOE Study
• Record relevant background details of
the process
• Define Response Variables
• Define Controllable Factors
• Define Held Constant Factors
• Define Uncontrollable Factors
Improve
1_05_01_062
Team selection for DOE's
"Who is involved in completing a problem statement and objective statement for an experiment"? asks the Black
Belt.
"Well, after completing the measure and analyze phases, all interested parties should contribute to the objective,
before other work on the experiment is begun. For example, an experiment to be designed and conducted in
manufacturing may be of interest to those in research or engineering. Having representatives from other
disciplines working on the objective of the experiment, can often result in a study that is more useful to a range
of interested parties. The makeup of the team should depend upon the nature of the problem. The team
selection process is crucial, because the team determines all aspects of the experiment", replies the Master
Black Belt
Define the “CT Characteristics”
Experiment Planning Sheets

Agent: _______________________ Department: ___________________ Process: ________________________
Experimental Date: ________________
Problem Statement:
Planning Sheet DOE Objective:
Page 1 Relevant Background:
List of Experiment Parameters

CT Characteristics Relationship to Objective Specifications
CT Characteristics Relationship to Objective Specifications
Web thickness on Estimate absolute Mean Nominal target

Wing Pickup difference from target + 0.000
Brackets and Standard Deviation - 0.002
Surface Finish Team has determined 10 - 20 Microns
Improve
that the surface finish ( Roughness )
1_05_01_063
should be as high a
quality as possible.
Defining the CT Characteristics
A Black Belt asks a Master Black Belt how he goes about choosing the CT Characteristics to be studied by an
experiment. "We usually have defined the CT characteristics during the measure and analyze phases, but what
makes a good response variable?" asks the Black Belt.
"This is step 3 in conducting a good experiment. In using a designed experiment, we always try to bridge the gap
between our current knowledge and experience on the process under study, to the knowledge that leads to
optimum operating conditions. As you said, the CT Characteristics for experiments, has usually been defined in
the measure and analyze phases, but there are a few things we should consider in their selection:
1. Where possible continuous variables should be selected for output responses, such as weight, diameter,
temperature, or concentration etc. The measurement units should be appropriate for the study, and they
may be relative units, such as percent of concentration by weight or by volume.
2. The response should capture as much information from the process as possible. For example, if the
experimental measure is a chemical bath temperature, it may matter that the temperature is taken from
a single position in the bath, or averaged over the surface area of the bath, or at different positions
throughout the bath.
3. Where possible, we should try to have a response that meets a target condition. Each experimental
condition will produce data that can be used to find conditions that will optimize the process output on
that desired target, with the minimum of variation.
4. To reduce measurement error, it is vital that for each CT characteristic, a gauge R&R study has been
carried out on measurement systems, prior to the start of the experiment", replies the Master Black Belt.
Define the “Control Factors”
Controllable Factors
Experimental Control
Factors
Normal
Settings
Proposed
Settings
How are they Set
Planning Sheet
Top of Page 2
Control Normal Proposed How are they

Factors Settings Settings Set
Feed Rate 3” per minute 1 Inch per Machine
minute and 5 Control pre-set
Inches per by the operator
minute
Air pressure 120 PSI 90 PSI and Operator
150 PSI control set
using a
calibrated
Improve
1_05_01_064
gauge
Define the "Control Factors"
"What should I think about when considering factors for an experiment"? asks the Black Belt.
"Factors chosen for the experiment are usually those that the Black Belt, and the team, believe will have the
greatest effect on the CT Characteristics. The selection, of important control factors, and estimation of the effects
these control factors will have on the response variables is very important. As with response variables it is
seemingly easy to generate a list of candidate control factors. In considering step 4 of conducting a good
experiment, all the potential factors that can be used in the experiment can be broken down into three types:
1. Control factors (discrete or continuous) thought to be very influential.

2. Factors that are held at a constant level and are controllable.
3. Factors that are uncontrollable, usually called nuisance or noise factors.
The control factors are chosen carefully, as they are the factors that we will use to vary the input settings of the
process throughout the experiment. We do this to determine which factor, or combination of factors, has the
greatest effect of centering our process on a target, while reducing variation. The wider the range of conditions
included in the experiment, the more generally applicable, will be the conclusions from the experiment", replies
the Master Black Belt.
Define “Held Constant Factors”
Controllable Factors
Control Normal Proposed How are they Set
Factors Settings Settings
Experimental
Planning Sheet
Bottom of Page 2
Factors Held at a Constant Setting
Constant Desired How to Control How are they Set
Factors Setting
Constant Desired How to Control How are they Set

Factors Settings
Temperature Conducted runs Machine to be running
of machine oil 88 to 96 0F. when machine is at 2 hours prior to start of
temperature experiment
Operators All operators to Use one “average” Chosen by supervisor
know how to operate operator in the ( For the experiment )
the process experiment
Improve
1_05_01_065
Define "Held Constant Factors"
"Is it important to think about factors that we can control and keep at a constant setting, but cannot vary in our
experiment", asks the Black Belt.
"Yes, this is a key aspect of designing any experiment. Many good experiments fail, because the team only
considered the control factors, they were going to change throughout the experiment. It is vital, that other factors
that you feel are important, but that you cannot vary are monitored closely, and held constant throughout the
experiment. The personnel carrying out the experiment must be made aware of this fact, and they must record
any occurrence where something changes, in one of these factors that are held constant. This may allow us to
determine a reason for any data points, that show up as special cause variation during our analysis of the
completed experiment", replies the Master Black Belt.
Define “Noise Factors”
Noise Factors
Experimental
Noise Strategy Employed to Control Noise Factors
Factors
Planning Sheet
Top of Page 3 Randomization & Replication
Conclusions from the Statistical Analysis
Noise Factors Strategy to Control Noise Factors

Ambient
0
temperature in the Make all runs below 70 F
Clean Room
Vibration of Ensure that no fork lift trucks pass by, or
Machine materials stacked during experimental
Improve
1_05_01_066
runs.
Define Noise Factors
"What do you do about factors that you cannot control, and hold constant during an experiment?" asks the Black
Belt.
"These factors are called noise, or nuisance factors. In many experiments, the variation due to background or
nuisance variables will be as great, or greater, than the variation due to the chosen factors. Generally a noise
factor, is a factor we think would have an effect on the CT characteristic, they can be grouped into three types,
those that are known and controllable, those that are known but uncontrollable, and finally those that are
unknown and therefore uncontrollable. We can use the blocking technique if we think a noise factor, that is
known and controllable, will have a significant effect on the CT characteristic. However, it important to try to
minimize the effects of all types of noise throughout the experimental sequence, and this usually achieved
through a complete randomization of the experimental sequence", replies the Master Black Belt.
Choosing Variable Levels
Y Factor Settings I think you can turn

True Effect off the water now.
The upper limit of
safety has been
reached!
Experimental Effect
Lo Hi
Factor Settings
(-) (+)
Y
True Effect
Experimental Effect
Improve
1_05_01_067
Lo Hi
Factor Settings
(-) (+)
Choosing the Variable Levels
"Once I have chosen my factors for the experiment, is there any tips you have for selecting factor levels for the
experiment"? asks the Black Belt.
"In step 5 of conducting a good experiment, the range of factor levels, selected for the study, will ultimately
determine the degree of belief in the data obtained as a result of the experi-ment. If we have a quantitative
variable with specifications, the high setting for the factors can be determined by computing approximately 20%
of the distance between the specifica-tion limits, and adding this amount to the upper specification limit. For the
low level, the 20% value is subtracted from the lower specification limit. If we have qualitative variables, with
many possible levels, then we should select levels using extreme values when possible. In the majority of cases,
unless you vary the factor levels enough, the chances of getting good experiment results are greatly reduced.
However, a key consideration in selecting factor levels, must be the safe operation of the experiment across all
the experimental conditions", replies the Master Black Belt.
"What about factors that are held constant throughout the experiment and nuisance factors"? asks the Black
Belt.
"Where possible, held constant factors should be kept at their nominal settings, and tightly controlled throughout
the experiment. As far as nuisance / noise factors are concerned, they differ from held constant factors in that
they cannot deliberately be kept at a constant level throughout the experiment, this may inflate the experimental
error, and make it difficult to assess the significance of the control factors. Where possible, if you can define
levels for a nuisance factor, then it may be possible to assess its impact, through the use of blocking". replies the
Master Black Belt.
Selecting the Experimental Design
Current Knowledge Type of Experiments
Little knowledge Fractional Factorials

(Screening Studies)
Some Knowledge Factorial Studies

(New levels, New factors)
Much Knowledge Confirmatory Full Factorial Studies
Improve
1_05_01_068
Selecting the Experimental Design
"Once I have established all the factors, and their associated levels, what is the next step in planning an
experiment", asks the Black Belt.
"Well! You are now ready for step 6, of conducting a good experiment, which is to select the experimental design
that will meet the stated objective of your study. The initial phases of DOE's will usually be screening studies, to
focus on finding the key factors affecting a process. These fractional factorial designs are used to examine five
or more variables in these initial studies. As we gain knowledge, experiments can be repeated using new levels
for factors previously studied, and some new factors. If we wish to study the relationships between the factors in
depth, two to four factors can be studied in a full factorial design. As new theories are developed, from
experience gained, a confirmatory experiment is usually run, in order to increase the degree of belief in the data
obtained from the previous experiments", replies the Master Black Belt.
Replication
Pressure : HHHH LLLL HHHH LLLL HHHH LLLL

Temp: HHLL HHLL HHLL HHLL HHLL HHLL
130
1st
Replication 120
2nd
Yiel d
Replication
110
100
In d e x 5 1 0 1 5 2 0 2 5
Improve
Test Sequence
1_05_01_069
Replication
"I know that replication refers to repeating aspects of an experiment, but what benefit does it give us"? asks the
Black Belt.
"Replication in a well designed experiment, is the primary means for analyzing the stability of factor effects, and
give us increased confidence in the results of the experiment. Replication provides a measure of the magnitude
of variation in the experiment due to nuisance variables. This minimizes the impact of noise variables, on the
main factor effects, by enabling nuisance variables to be averaged out", replies the Master Black Belt.
"Surely the amount of replication in an experiment, directly affects the amount of resources needed (time,
budget, materials, etc.) for the study", asks the Black Belt.
"As a guideline, the initial experiment should consume about 25% or less, of the resources allocated to the total
DOE study. However, it's a fact of life, that many studies will have budgetary constraints. Your costs in PETMET
should reflect the full cost of all your proposed DOE cycles. But always remember the most important
consideration in conducting a DOE, is not the number of replications per level of a factor, but the breadth of
conditions under which the comparisons are made. Replication, over a small range of factor settings, will not
give us confidence in our experimental results. For example, DOE runs on a milling machine carried out on
different days, and made from different material batches, would be preferred to a study based on consecutively
produced units which in DOE terms is called repetition. As a rule of thumb, it is desirable to carry out between 2
to 5 replications for each run", replies the Master Black Belt.
Randomization
Run A B C D Block Reac Temp Time Cat Run Run

Order 1 2 3 4 No. A B C D Day Night
1 -1 -1 -1 -1 2 1 230 30 Std 8
2 1 -1 -1 -1 1 2 230 30 Std 8
3 -1 1 -1 -1 1 1 300 30 Std 7
4 1 1 -1 -1 2 2 300 30 Std 1
5 -1 -1 1 -1 1 1 230 60 Std 4
6 1 -1 1 -1 2 2 230 60 Std 13
7 -1 1 1 -1 2 1 300 60 Std 7
8 1 1 1 -1 1 2 300 60 Std 3
9 -1 -1 -1 1 1 1 230 30 New 6
10 1 -1 -1 1 2 2 230 30 New 3
11 -1 1 -1 1 2 1 300 30 New 2
12 1 1 -1 1 1 2 300 30 New 5
13 -1 -1 1 1 2 1 230 60 New 6 Randomized
14 1 -1 1 1 1 2 230 60 New 2 sequence for both
15 -1 1 1 1 1 1 300 60 New 1 experimental
16 1 1 1 1 2 2 300 60 New 5 blocks
Improve
1_05_01_070
Blocking disallows the given variable from influencing the primary response variable.
Randomization
"I notice in Minitab, that standard order of the experimental runs is not in the order shown in my textbooks, why
does this happen?" asks the Black Belt.
"In producing any design in Minitab, the runs are automatically randomized, unless you elect to turn off this
function. Randomization, is a tool that addresses the problem of noise (nuisance) Factors. Noise Factors are
process variables that affect the CT Characteristic but have not been identified by the Black Belt or have been
identified, but are not controllable, or have been identified and are controllable. Typical nuisance variables are
environmental effects such as temperature or humidity, batch-to-batch variation in raw materials, the effect of a
machine warming-up, or position of parts in an oven etc".
"When blocking is employed, the randomization of the DOE is carried out in Minitab within each block, rather
than over all runs in the design. For example, an experiment with a block based on day and night shift, would be
randomized to assign combinations of factors to be carried out during each shift. In some other cases, a random
order for the conduct of a test could be prohibitively expensive. For example, changing the level of one particular
factor might require a machine to be shut down for most of the day, whereas changing all the other factors could
be done in a few minutes. In cases where it is difficult to fully randomize the design, a great deal of care should
be taken in interpreting the results of the experiment".
"A rule of thumb with respect to randomization for analytical studies is, after restricting important background
variables in the form of blocking, randomize in all remaining situations unless the constraints to randomization
have been considered, and found to be prohibitive", replies the Master Black Belt.
Collecting the Data
Key tasks in Collecting Data:

• Ensure that the experiment is run in a randomized
sequence, as defined by the DOE design. (Ensure
the design has been randomized in Minitab).
• Ensure that if Blocking is involved
in the design, the runs are
randomized in accordance with
the blocks.
DA
• Reset each controllable factor as
TA
accurately as possible after each
experimental run.
• Hold all other factors as stable as possible during the experiment.
• Measure the CT characteristics as accurately as possible.
Improve
1_05_01_071
• Record any unusual occurrences that happen when running the

experiment.
Collecting the Data
"Are there any things I should take into account when running the DOE, and collecting the data?", asks the Black
Belt.
"Collecting the data is step 7 of conducting a good experiment, they say that there is no such thing as a poor
experiment, only a poorly designed or poorly executed one. Poor planning leads to poor execution. It is vital that
the person, or persons, who will be carrying out the DOE and setting the factor levels, are fully conversant with
the need for rigor and attention to detail, throughout the experimental sequence. A way to ensure that this rigor
and attention to detail is covered, is to run a pilot study, to verify and improve the data collection procedures,
before starting the actual experimental sequence. The operators running the process, are a key component for a
successful experiment, not only will they have to run the experiment in the randomized sequence, they will also
need to record in detail any unusual occurrences at are present during the experiment. They will need to re-set
the control variables, then hold them constant, and usually measure the output from each experimental run. I
cannot over emphasize the need for highly capable measurement equipment, as the results of the experiment
can be greatly affected due to a large degree of measurement error. So, the accurate setting of controllable
factor levels, are another aspect in successful experimentation, that needs to be carefully monitored throughout
the experimental cycles", replies the Master Black Belt.
Analyzing the Data (ANOVA)
Create a No Check the reason for

General the non-normality in
Linear Model Does the residual
the residuals and re-
ANOVA model fit the
run that experimental
assumptions of
combination if
normality?
required
Yes
Create Main
Identify the vital few Effect Plots
factors with
(Pareto or Pie Chart)
Investigate
Improve
and or the Normal
1_05_01_072
Probability Plot of Effects Interaction Plots
Analyzing the Data (ANOVA)
"What steps should I follow when I analyze my experimental data?", asks the Black Belt.
"Analyzing the data is step 8 of conducting a good experiment. The first task in analysis is to construct an
ANOVA table for the full model in Minitab. I then store the "Residuals" and "Fits". Using the function Stat >
ANOVA > Residual Plots, and ensure that the residual model fits all the assumptions of normality. After
producing the ANOVA table, and checking the residuals I always like to determine if the results of the ANOVA
analysis have any practical significance", replies the Master Black Belt.
"What do you mean by practical significance, surely we have determined all factors and interactions that are
significant from the ANOVA table?" asks the Black Belt.
"It is possible for something to be statically significant, but be of little practical use in improving the process. So,
we analyze Sum of Squares in the ANOVA table, to produce a Pareto chart or pie chart. We can also produce a
"Normal Probability Plot for Effects", and from analyzing these charts we can determine what practical
significance each main effect, and interaction effect, has on the process", replies the Master Black Belt.
"Are there any more steps in analyzing the data?" asks the Black Belt.
"Yes, after that we produce main effect plots and interaction plots using the functions, Stat > ANOVA > Main
Effect plot, for a graphical interpretation of main effects, and Stat > ANOVA > interaction Plot, for graphical
interpretation of two-way interactions", replies the Master Black Belt.
Conclusions and Recommendations
This way to an Optimized Process
• Check all observed data for

accuracy.
• Explain results of the experiment in
terms of graphs as well as numbers.
• State results in practical terms as
well as statistical terms.
• Limit conclusions to summary
evidence taken directly from the
experiment.
Improve
1_05_01_073
• Point out the way ahead, if further

experiments are required.
Conclusions and Recommendations
"Once we have analyzed the data, and have determined the practical significance of each factor and interaction,
we determine the combination of factors and interactions, that optimize our process to meet our CT objectives. It
is good practice to replicate the optimum set-up, and ensure that the results can be reproduced. When we have
proven our findings, we have to draw our conclusions, and make our recommendations to senior management.
This is step 9 in conducting a good experiment", says the Master Black Belt.
"What steps do you follow in drawing your conclusions?" asks the Black Belt.
"Some key points are to check, that you have considered all the observed data. Confine all initial conclusions
and deductions, to the experimental evidence at hand. As we will need to explain the results of the study, and
gain buy-in from the Champion, and the owners of the process, always try to explain the analysis in both
graphical as well as numerical terms."
"From this data, explain the results / conclusions in terms of practical meaning, as well as their statistical
significance. I have found out, that it is vitally important to point out to the Champion, and the personnel who
work on the process, the results of the study, the implication of the results, and if there is a need for further
work".
"I always write a report clearly describing the experiment work, giving the background to the study, pertinence of
problems, and the meaning of the results. I find it of great benefit to focus on using tabular and graphic methods
of presenting data. A rule I always try to adopt, is to supply sufficient information to permit readers of the report
to verify results, and to draw their own conclusions, but limit conclusions to objective summary of evidence
provided by the study", says the Master Black Belt.
Achieve the Objective
• When the process has been

optimized it is time to
institutionalize the changes.
• Control key CT Characteristics
using methods outlined in the
control phase.
• Control key process control
factors using methods outlined
in the control phase.
• Keep the process operating at
that the peak of its
Improve
performance.
1_05_01_074
Achieve the Objective
"What do we do, if we find that we have not optimized the process from our experiment?" asks the Black Belt.
"Well, usually it is impossible to fully optimize a process based on one experimental sequence. You will
remember from your training, that it was recommended that the first experiment consume only 25% of the
budget and resources. It has been my experience, that you always find something from an experiment that no
one knew about the process, this knowledge, usually points out the direction that the Black Belt should go in
setting factor levels for another experiment." replies the Master Black Belt.
"When we have proven that the process is improved, and is set at the optimum operating configuration, what
comes next?" asks the Black Belt
"If you have reached this stage in your experimental process, then you are ready to implement step 10 of
conducting a good experiment. You are now ready to institutionalize the changes, to ensure that entropy does
not affect the optimized process, over time. Entropy, is the fact that everything in nature decays. We can improve
and optimize the process but if we do not institutionalize the recommended changes, formulated by the DOE's,
and put controls on the key aspects of the process, it will revert back to a less than perfect process, over time.
This is the main reason for the control phase, using control techniques, such as SPC to control the critical to
process parameters (CTPs)", replies the Master Black Belt.
Lessons Learned
• In order to design an experiment, we must first clearly understand the process and decide what feature it is
that we want to investigate. Once the experiment is completed, additional understanding of what is important
will be gained and another experiment can be designed. We do this until the desired result is obtained.
• Each DOE has two elements: the "Experimental Design" and the "Analytical Approach". The two elements
must be well thought through in order to collect appropriate data that can be meaningfully analyzed.
• Compared to the traditional one-factor-at-a-time approach to experimentation, a DOE can provide a wealth
of knowledge about several factors (and their interactions) in a very cost effective manner because several
factors can be varied at the same time, all within one experiment.
• The most common way to design an experiment (so that all combinations are considered) is to use Yates
Standard order. This notation (-1, 1) is used to set factor levels and coefficients. The Contrast concept is
used in statistical analysis to determine if a design is balanced and to determine orthogonality.
• The ANOVA table is used to assess statistical significance initially and then practical significance of factor
effects and interaction effects on the output. A factor may be statistically significant, but practically
insignificance.
• We use pie charts and Pareto diagrams to visually display the relative effects of several factors. We then
often plot the main effects on a graph in order to visualize the difference in the response output between the
low and high settings of each important factor. Graphs are also useful to visualize interaction effects
between factors and to find the optimal settings for both factors.
• Even if the main effect is not statistically significant, it may affect the output through an interaction with
another factor.
• We should consider both the main effects and interaction plots prior to specifying optimal settings.
• Two factor interactions are common, but three factor and higher interactions are rarely seen in our industry.
• It is common practice to do a fractional factorial experiment when we have five factors or more.
• In general, factor levels should be set wide, but not so much so that they extend beyond the operational
range of the factor.
• The general steps for conducting a statistical analysis are ANOVA (with residual analysis), practical
significance (sum of squares, pie chart, etc.) and main effect plots (with interactions considered).
• A successful experiment does not happen by chance, it takes careful planning and attention to detail to
ensure that all designed experiments produce meaningful results. There is no such thing as a poor
experiment, only an experiment that is poorly designed or executed.
• Remember, actively communicate your plan with all those concerned is vital to the success of an
experiment.

I1 - Experiment Design Tools

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

I1 - Experiment Design Tools

Загружено:

Авторское право:

Доступные форматы

Six Sigma

Experiment Design Tools

Experiment Design Tools

• What are the guiding principles surrounding statistically designed experiments?

• How can conflicts between competing performance objectives be resolved?

TO IDENTIFY THE VITAL FEW DO AS EARLY AS POSSIBLE IN

Focus on defect detection

Product Process Manufacturing Field

Experiment Design Tools 3

Fundamental Principles of Experimentation

Experimentation - the manipulation of controllable factors ( indepen -

1. Factor A affects the mean 2. Factor B affects the standard

3. Factor C affects the mean and 4. Factor D has no effect

Input Factors (Xs) Output (Ys)

Experiment Design Tools 6

Sources of Experimental Factors

Experiment Design Tools 7

Types of Experimental Factors

What is "Design of Experiments" (DOE)

What is "Design of Experiments" (DOE)

Flow Rate of Flow Rate of

Water Temperature = f (flow of hot water, flow of cold water)

Experiment Design Tools 9

The Shower Experiment

1. Experimental Design Questions:

Questions: 2. Analytical Procedure

How will we find relationships? • Regression

analysis have to be?

Experiment Design Tools 10

2. Analytical Procedure: A means to extract relevant information from the experiment.

Why Not One-Factor-at-a-Time?

Benefits of Statistical Experiments

Time to Very Long

Very High Rel

Confidence Very Very

Conclusion: Statistical Experiments are Rich With Information!!

Benefits of Statistical Experiments

Very • Trial and Error Methods

Formal – Modeling, Multiple Factor Levels, Optimization

Experiment Design Tools 13

Screening Characterization Optimization

Experiment Design Tools 14

Experiment Design Tools 15

Terms Used in DOE

• DOE – Design of Experiment, or "an invitation for an information event to occur".

Experiment Design Tools 16

Main Effects and Interactions

DESIGN MATRIX Vectored

Experiment Design Tools 17

Balanced ΣX i = 0 for each factor sum

0 for all dot product pairs

Experiment Design Tools 18

Experiment Design Tools 19

Run Order for a DOE

Response Variable (Y)

Experiment Design Tools 20

How to Select Factor Levels

• Draw on all past experience to provide knowledge of your process.

Experiment Design Tools 21

Barriers to Effective Experimentation

The Car Mileage Experiment

Measure : Litres/100 Km. E. Air Conditioner Off On