The Theory of Statistical Inference Using Hypothesis Testing

Written by Mary Newman

Scope
This paper educates readers about statistical inference for the first time. It could be shared with anyone who does not have an understanding of the math and science behind statistics, such as individuals who create a product that needs to be tested. Anyone that has taken a basic statistics course will already be familiar with the information provided. Hence, the intent is to introduce the information to first time learners or to provide a refresher for someone who may not recall the basics from an introductory statistics course.

Introduction
Statistical inference is the process of drawing conclusions about a large population from sample data that is subject to random variation. Statistical inference is used in a variety of disciplines to test hypotheses about populations. A drug company may use statistical inference to see if a drug meets FDA requirements, or a manufacturing company may use statistical inference to test for defective products. Statistical inference tells us what we can say about a populations parameters through analyzing data. For instance, we can estimate the true success rate of a drug or the true proportion of defective products. By analyzing data with statistics, we can draw conclusions that we believe to be closest to the true parameters. The information provided here will explore how statistical inference works by starting with general topics and definitions before moving into specifics.

* Technique: Sampling

When we use statistics, we are trying to draw conclusions about a population. If we had data for every individual in a population, known as a census, we could easily determine the parameters.

Because populations can be impractically big to census, statistical inference uses sampling to estimate parameters. By taking a sample from a population, we can estimate what the true parameters might be with a hypothesis test. See Figure 1.

Figure 1.

Population Distributions
Every population follows a distribution, or a probability pattern. Looking at data patterns thoroughly allows us to estimate probabilities for a population. We describe the patterns with a type of distribution using random variables. For instance, a random variable has a uniform distribution if it has equal probability of attaining all possible values.

While not all probabilities are as clear as rolling a die, we can estimate distributions by looking at data. We say a population is approximately normal if the data follows the empirical rule; that is if approximately 68% of the observations fall within one standard deviation of the mean, 95% fall within two standard deviations, and 99.7% fall within three standard deviations. The normal distribution is also commonly known as the bell curve. See Figure 2.

Figure 2.

The Sampling Distribution

Normal distributions are the most useful, because it turns out that all sampling distributions approach normality as you take bigger samples. A sampling distribution describes the probabilities of a statistic from all possible samples of the same size.

It turns out that it does not matter what type of distribution a specific population follows, its sample distribution will always approach to normal. This is because of the Central Limit Theorem, which states that under given conditions, the sampling distribution of a population will tend to normal with a sufficiently large sample size. It turns out that a sample size of 30 or more is considered sufficiently large and will have an approximately normal sampling distribution. See Figure 3.

Figure 3.

The Central Limit Theorem is what allows for statistical inference. Since we know how a normal distribution behaves, we can start to make inferences based on a sample of any population. The distribution of the population itself doesnt matter; its sampling distribution will always become approximately normal. The sample mean becomes an observation of the sampling distribution, which means we can estimate the likelihood of getting statistics, such as a sample mean, from a sample using a hypothesis test.

* Hypothesis Test

A hypothesis test compares two hypotheses about a parameter, and determines whether or not to reject or accept the null hypothesis. It typically takes five steps: 1. 2. 3. 4. 5. Determine null and alternative hypotheses to test Set significance level Calculate test statistic and rejection region Evaluate the hypotheses Calculate the p-value

1. Determine null and alternative hypotheses to test The null hypothesis (Ho) is what we think the parameter is. For instance we could say Ho is that the average height of all women is 64 inches, or = 64, where is the true average. The alternative hypothesis (Ha) would be the opposite of the null, or that the average height of all women is not 64 inches. We can therefore choose to hypothesize 64, > 64, or < 64 for Ha.

2. Set significance level The significance level is the percent of the samples we will reject because they are too far away from the mean hypothesized by the null. The most common significant levels are 1% and 5%. If we reject a hypothesis at the 5% level, this means that if the null hypothesis is true, there is less than a 5% chance that we would get a sample with the statistics found from the data. Choosing a significance level implies how precise we want to be. The higher the significance level, the more samples we will reject.

3. Calculate test statistic and rejection region After collecting data from a sample, we can calculate the sample statistics and compare it to the null hypothesis by using a standard normal variable (Z). A standard normal variable is the difference between the sample statistic and the hypothesis parameter divided by the standard deviation of the population. Every observation from a normal distribution has a corresponding Z-score used to calculate probability. The rejection region also needs to be determined. By finding the corresponding values that contain the significance level, we set a range where we will reject the null hypothesis. For instance if the significance level is 5%, we will reject any samples that gives a statistic 1.96 standard deviations away from the mean. These sample will have less than 5% probability under the null hypothesis, and are therefore significantly unlikely. See Figure 4.

Figure 4.

4. Evaluate the hypotheses The test statistic and rejection region values are compared to the null hypothesis. If we get a test statistic that is within the rejection region, we reject the null hypothesis. If we get a test statistic that is not within the rejection region, we fail to reject the null hypothesis. It is important to note that failing to reject the null, does not imply accepting it. It simply means that the data does not provide significant data to suggest the null is false.

5. Calculate the p-value The p-value is another way to evaluate the hypotheses. Using the normal probability distribution function the probability of getting a sample with its specific statistics can be calculated. Since the function is rather complex, a z-table lists the values for the standard normal variable, which allows us to get the probability easily.

Possible Error

After drawing a conclusion from a hypothesis test, possible errors must be considered since there is no way to know the parameters for certain. In a hypothesis test there are two types of errors: Type I error: rejecting the null hypothesis when it is in fact true. Type II error: failing to reject the null hypothesis when it is false. Type I error occurs when we get a test statistic in the rejection region. Since the sampling distribution includes all possible samples, the samples that we choose to reject are still possible. Although they are significantly unlikely according to the chosen significance level,

they are still attainable under the null, and it could be the case that we reject the null falsely. Type II error occurs we fail to reject a test statistic under the null hypothesis, but in reality we should reject it. This would be the area under the alternative hypothesis that we would not reject under the null. In Figure 5, we see that Type I error is the red area with white stripes, and Type II error is the blue area with white stripes.

Figure 5.

Conclusion

Statistical inference allows us to draw conclusions about populations of all kinds. It is especially useful when trying to evaluate large and unattainable populations. Whether it be drug test results or batting averages, understanding the basic theory behind the analysis can help individuals in all fields better understand their data.

Picture Citations Cover photo: Cartoon pie chart. Cartoon. n.d.. Interesting VoC Derived Statistics. Voice of the Customer, Customer Experience & Big Data Blog. Web. 6 Mar. 2014. Figure 1: Diagram of sample from population. Diagram. n.d. Customer Satisfaction Surveys Defining Survey and Census for Your Successful Survey. Snap Surveys. Web. 6 Mar. 2014. Figure 2: Empirical Rule. Graph. 5 Jul. 2012. Spc tools-empirical rule. Index of Images. Web. 6 Mar. 2014. <http://threes.com/cms/images/stories/mathematics/spctools-empiricalrule.gif> Figure 3: t distribution approaches normal distribution. Graph. n.d. Students t-distribution. Pink Monkey. Web. 6 Mar. 2014. Figure 4: Rejection region. Graph. n.d. Statistical Testing Using Rejection Region Approach. Stat 500 Applied Statistics. Web. 6 Mar. 2014. <https://onlinecourses.science.psu.edu/stat500/node/44> Figure 5: Type I and Type II error. Graph. n.d. Web. 6 Mar. 2014. <https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcRauurQ1THFvh4W_He DiKyjZnI85Qp4VUl7NsvQAME4TDg9ShLSew>