The Power of a Statistical Test

• Definition
• Ways to Increase the Power
• Two Types of Error Revisited

Power of a test July 29, 2008 -1-

 The power of a statistical test

• The power of a significance test measures its ability to detect

an alternative hypothesis.
• The power against a specific alternative is calculated as the
probability that the test will reject H0 when that specific al-
ternative is true.
• This calculation requires knowledge of the sampling distribu-
tion of the test statistic under the alternative hypothesis.

Power of a test July 29, 2008 -2-

 Example: Does exercise make strong bones?

Can a 6-month exercise program increase the total body bone min-
eral content (TBBMC) of young women? A team of researchers is
planning a study to examine this question.

Based on the results of a previous study, they are willing to assume

that σ = 2 for the percent change in TBBMC over the 6-month
period. A change in TBBMC of 1% would be considered impor-
tant, and the researcher would like to have a reasonable chance of
detecting a change this large or larger.

Are 25 subjects a large enough sample for this project?

Power of a test July 29, 2008 -3-

 1. State the hypotheses: let µ denote the mean percent change.

H0 : µ = 0
Ha : µ > 0

2. Calculate the rejection region: The z test rejects H0 at the

α = 0.05 level whenever
x̄ − µ0 x̄
z= √ = √ ≥ 1.645
σ/ n 2/ 25
that is, we reject H0 when x̄ ≥ 1.645 √225 = 0.658
3. Compute the power at a specific alternative: The power of
the test at alternative µ = 1 is

P (x̄ ≥ 0.658|µ = 1)
x̄ − µ 0.658 − 1
= P( √ ≥ √ |µ = 1)
σ/ n 2/ 25
= P (Z ≥ −0.855)
= 0.80

Power of a test July 29, 2008 -4-

 Ways to Increase the Power

• Increase α. A 5% test of significance will have a greater chance

of rejecting the alternative than a 1% test because the strength
of evidence required for rejection is less.
• Consider a particular alternative that is farther away from µ0.
Values of µ that are in Ha but lie close to the hypothesized
value µ0 are harder to detect than values of µ that are far from
µ0 .
• Increase the sample size. More data will provide more infor-
mation about x̄ so we have a better chance of distinguishing
values of µ.
• Decrease σ. This has the same effect as increasing the sample
size: it provides more information about µ. Improving the
measurement process and restricting attention to a subpopu-
lation are two common ways to decrease σ.

Power of a test July 29, 2008 -5-

 Example: Does exercise make strong bones? (cont.)
• Change the significance level to α = 0.01.
The z test rejects H0 at the α = 0.01 level whenever
x̄ − µ0 x̄
z= √ = √ ≥ 2.32
σ/ n 2/ 25
that is, we reject H0 when x̄ ≥ 2.32 √225 = 0.928
The power of the test at alternative µ = 1 is
x̄ − µ 0.928 − 1
P (x̄ ≥ 0.928|µ = 1) = P ( √ ≥ √ |µ = 1)
σ/ n 2/ 25
= P (Z ≥ −0.18)
= 0.57

• Change the alternative to µ = 2.

The power of the test at alternative µ = 2 at α = 0.01 level is
x̄ − µ 0.928 − 2
P (x̄ ≥ 0.928|µ = 2) = P ( √ ≥ √ |µ = 2)
σ/ n 2/ 25
= P (Z ≥ −2.68)
= 0.996

Power of a test July 29, 2008 -6-

 • Increase the sample size n = 100.
The z test rejects H0 at the α = 0.01 level whenever
x̄ − µ0 x̄
z= √ = √ ≥ 2.32
σ/ n 2/ 100
2
that is, we reject H0 when x̄ ≥ 2.32 √100 = 0.464
The power of the test at alternative µ = 1 is
x̄ − µ 0.464 − 1
P (x̄ ≥ 0.464|µ = 1) = P ( √ ≥ √ | µ = 1)
σ/ n 2/ 100
= P (Z ≥ −2.68)
= 0.996

• Decrease σ = 1
The z test rejects H0 at the α = 0.01 level and for n = 25
samples whenever
x̄ − µ0 x̄
z= √ = √ ≥ 2.32
σ/ n 1/ 25
that is, we reject H0 when x̄ ≥ 2.32 √125 = 0.464
The power of the test at alternative µ = 1 is
x̄ − µ 0.464 − 1
P (x̄ ≥ 0.232|µ = 1) = P ( √ ≥ √ |µ = 1)
σ/ n 1/ 25
= P (Z ≥ −2.68)
= 0.996

Power of a test July 29, 2008 -7-

 Two Types of Errors Revisited

Recall that the following four outcomes are possible when conduct-
ing a test:

Our Decision
Reality
H0 Ha
√
Type I Error
H0
(Prob = 1 − α) (Prob = α)
√
Type II Error
Ha
(Prob = β) (Prob = 1 − β)
• The significance level α of any fixed level test is the probability
of a Type I error.
• The power of a fixed level test against a particular alternative
is 1 − β for that alternative.
• In practice, we first choose an α and consider only tests with
probability of Type I error no greater than α.
• Among all level α, we select one that makes the probability
of Type II error as small as possible (i.e. the most powerful
possible test).

Power of a test July 29, 2008 -8-