Developing and testing

a hypothesis

Learning material:
Chapters 9-11: Confidence Interval
Estimation
Chapters 12-14: Testing hypotheses

Learning Objectives

Outline sources of ideas/hypotheses

Understand the process of hypothesis testing

Understand the purpose and interpretation

of confidence intervals

Ideas and evidence

Epidemiology
Biology
Case
reports

Clinical
observation

IDEAS

Imagination
Reasoning

Hypotheses

(Proposed answers)

}
Experimental studies

Observational studies

Tests of
hypothesis

EVIDENCE-BASED INFORMATION
4

Examples:
Mechanisms of disease at the molecular level
Drugs against antibiotic-resistant bacteria are developed through
knowledge of the mechanism of resistance

Case reports-Clinical observation

Tamoxifen, developed for contraception, was found to prevent
breast cancer in high-risk women.

Beliefs about herbal remedies

Aspirin is a naturally occuring substance that has become
established as orthodox medicine after rigorous testing.

Epidemiologic studies of populations

Observations of low prevalence of colonic diseases (irritable
bowel syndrome, appendicitis, colorectal cancer) in Africa
(where diet is high in fiber) led to efforts to prevent bowel
disease with high-fiber diets.
Comparisons across regions have suggested the value of
fluoride to prevent dental caries.

Learning Objectives

Outline sources of ideas/hypotheses

Understand the process of hypothesis

testing

Understand the purpose and interpretation

of confidence intervals

When to apply Statistical Hypothesis Testing ?

- To answer a research question
Example
Is new antibiotic X a more effective drug for treating
infections than standard treatment Y?
To test the folowing hypothesis:
proportion of survivors in population of patients treated by X=
=proportion of survivors in population of patients treated by Y

Important terms in Hypothesis Testing

Null and Alternative Hypotheses

P-value
Type I and Type II Errors
Significance level
Confidence level
Power of a test
Test statistic
One(two)-tailed test
Statistical tests

Null and Alternative Hypotheses

The null hypothesis (H0) is usually of the form:
there is NO difference between the two groups.
The alternative hypothesis (Ha) would be:
there is a difference.
Examples of H0:
proportion of survivors in population of patients treated by X=
=proportion of survivors in population of patients treated by Y:
H0: X= Y
mean weight in population of patients in your clinic=mean weight in
general population: H0: =0

The p-value
Example
H0 : The coin is fair (i.e. will produce as many heads as tails)
=> population proportion =0.5
Outcome: 42 heads out of 100 times tossing =>
sample proportion p=0.42.
Is the difference between 0.5 and 0.42 statistically significant
or due to chance? How do we decide what proportion of heads
we might expect to get if the coin is fair?
Matter of interest: P(heads 42 | coin is fair)=0.067 > 0.05
=> the coin is most likely fair

p-value
P(heads 41 | coin is fair) = 0.044 < 0.05
=> the coin is most likely not fair

p-value
if outcome: 41 heads

10

Definition
The p-value is defined as the probability of obtaining the
result, or a more extreme result, if the null hypothesis is true.
Interpretation of p-value:
We usually say, if p 0.05, then the results are
statistically significant (or unlikely due to chance), and
there is strong enough evidence against the null hypothesis.

11

Example
Odds Ratios of antimicrobial resistance in hospitalacquired versus community-acquired infections

<0.05
>0.05
For Ceftazidim the difference in Odds Ratios for Denmark (OR=15.3)
and Israel (OR=3.4) is statistically significant (p<0.05).
For Ceftriaxone there is no evidence to conclude that the difference
(between OR=8.0 and OR=3.7) is statistically significant (p>0.05).

13th ICID, Abstract 66.004

12

Type I and Type II Errors

Type I error (false positive): Rejecting H0 when it is true;

(alpha): the probability of Type I error;
Significance level (the probability of Type I error that we are
willing to accept): usually = 5%;
Confidence level: 1- , usually 95%.
Type II error (false negative): Accepting H0 when it is false;
13
: the probability of Type II error.

Power

The probability of finding a specified difference, or larger,

when a true difference exists
(the probability of correctly rejecting the null hypothesis).

Power = [1 - (probability of a Type II Error)] =1-

14

To define Power
Power depends on:
significance level ()
sample size
minimum size of the effect that would be clinically
useful
Fix (= 5%) and then try to minimize (maximize 1 - )
Keeping other variables constant, the best way to increase
power is to increase sample size
(Studies with small sample sizes tend to have low power)
Typical power is 80% or greater
You can compute the power for a study that you are about
to run (good idea!), or you can compute post-hoc power
after an experiment is done.
15

Example

RCT of homeopathic arnica for post fracture healing

Conclusion:

16

> 0.05 (= )
probability of observing 5% difference or more,
when the null hypothesis is true
p>5% => do not reject null hypotesis
17

=> do more research, with a larger sample

18

Test statistic

A function of the sample data on which the

decision is to be based.
Follows the general format:
Observed Value - Expected Value (H0 )
Standard Error

19

Test statistic: Example

H0: = 0
one sample t-test statistic

x 0
t
s/ n

difference in means

difference in means
variability

statistical difference

0
variability

20

Rejection area
H0: = 0
one sample t-test statistic

Sampling distribution of t under H0

x 0
t
s/ n
t*

critical value

The t-distribution contains all possible values of t computed for

random sample means selected from the population described by H0.
The rejection region is the set of all values of the test statistic that
cause us to reject the null hypothesis.

One(two)-tailed statistical test

One-tailed

Ha: >0

=.05

Two-tailed
Ha: 0
=.025

=.025

22

The hypothesis testing process

Use your research question to define H0 and Ha.

Collect samples data and determine sample statistics (e.g., sample mean,
sample proportion,).

Choose the appropriate statistical test.

Check assumptions for this test.

Express H0 and Ha in mathematical terms.

Decide on a level of significance (usually 5%).

Calculate test statistic.

Refer test statistic to known distribution it would follow with H0.

Calculate p-value (a probability of a test statistic arising as or more extreme

than observed, if H0=true).

If p-value < significance level then reject H0; otherwise do not reject H0.
23

Statistical test
There are many different statistical tests. The choice
of which test to use depends on several factors:

The type of data (continuous, nominal, etc.)

The distribution of the data (normally distributed
or not)
The type of sample statistics (mean, median,
proportion, ratio, etc.)

24

One Sample t test

Purpose: to compare the mean score of a sample to a known

value.

Assumptions:
Random sample
Metric sample data
Normally distributed sample data
(although this assumption is less critical when the sample size 30)

H0: = 0
test statistic=

observed value - expected value (H 0 )

standard error

x 0
t
s/ n
25

One sample t test - example

Research question: Are patients at your clinic have higher

level of blood creatinine than the national average?

H0: creatinine level in population of men in your clinic =

creatinine level in general male population.
For healthy men creatinine levels are normally distributed,
the average creatinine level in male population is 0=0.9 mg/dL.

 Sample:
_

random sample
metric data

x 1.2 mg / dL
s = 0.352 mg / dL
se= 0.088 mg / dL

checking normality:

 Test: one-sample t-test

a=5%
H0: =0.9; Ha: 0.9

2-tailed test
Test statistics: t = (1.2-0.9) / 0.088 = 3.4
(=> the results are 3.4 standard error units higher the expected value).
Use a statistic table to look up the t-distribution and find
critical value t*.

28


df

t distribution depends on
degrees of freedom: df = n-1

29

t*=2.13 (depends on df=n-1=15)

Calculate p-value:

P-value is the area in

the tails greater than |t|=3.4

t=3.4

t=-3.4

Here p < a=0.05

t*

P-value < a

=> reject the null hypothesis.

We conclude that the result we saw was unlikely to have

occurred by chance alone, and that difference in mean
creatinine levels in males from the clinic and in a general male
population is statistically significant (0.05 significance level).
30

One sample t test in SPSS

p<5% => Reject Ho

Difference is statistically significant!

31

Learning Objectives

Outline sources of ideas/hypotheses

Understand the process of hypothesis testing

Understand the purpose and interpretation

of confidence intervals

32

Alternatively: Estimation

Population

Mean, , is
unknown

Random Sample

Mean
X = 1.2

I am 95%
confident that
is between
1.01 & 1.38.

Sample

CI = an interval of values that contains the true value

of the parameter with specified confidence.

95% CI for a population mean

(If is known)
Assumptions

Population standard deviation () is known (this is hardly

ever true)
Population is normally distributed or sample is large

95% Confidence Interval Estimate:

X 1.96
n

34

95% CI for a population mean

( If is unknown)
Assumptions

Population standard deviation ( ) is unknown => use

sample standard deviation (s)
Population is normally distributed or sample is large

95% Confidence Interval Estimate:

X t * s
n

Use a statistic table to look up the t-distribution and find critical

value t*.

35

CI of the Mean in SPSS (creatinine example):

The 95% confidence

interval does not
include the comparison
value (0.9).
This is equivalent to
rejecting the null
hypothesis.
36

Graphical presentation of a CI

37

CI for a Mean Difference

CI of the Mean Difference does not include 0

<=> p-value<0.05
=> reject H0
38

CI for an Odds Ratio

Odds Ratios of antimicrobial resistance in nosocomial versus
non-nosocomial infections

In Denmark the CI of the OR contains 1 (0.8-18)

=> despite a sample OR of nearly 4, there is no evidence to conclude
that hospital acquisition of a coagulase-negative staphylococci in
Denmark is a statistically significant risk for resistance to penicillin.
In Israel the CI of the OR does not contain 1 (1,3-3.8)
=> hospital acquisition of a coagulase-negative staphylococci in
Israel is a statistically significant risk for resistance to penicillin.

13th ICID, Abstract 66.004

39

CI of the OR in SPSS (Apgar score example):

40

Learning Objectives

Outline sources of ideas/hypotheses

Understand the process of hypothesis testing

Understand the purpose and interpretation

of confidence intervals

41

42