CHAPTER 3 EXPERIMENTAL AND SAMPLING DESIGN Overview of how to answer a research question: 1.

. Pick a specific question you want to answer. 2. Decide on your population. 3. Select a sampling design and gather your sample. The choices for sampling designs for this class are: voluntary response (the only one not random, not the best) simple random sample stratified random sample multistage sample catch-and-release sample 4. Decide whether to conduct an observational study or an experiment. If observational study, just state the sampling design. If experiment, the choices for experimental designs for this class are: completely randomized design block design matched pairs 5. Choose your response variable and your explanatory variables. Decide on your treatments (for experiments). 6. Collect the data. 7. Analyze your data using either exploratory data analysis (looking for trends/relationships in the actual data) or formal statistical inference (answering statistical questions with a known degree of confidence). 8. State your conclusions. What can go wrong? Bias (response bias, nonresponse, undercoverage) Variability Poor experimental design (not using a control, not randomizing, not replicating) Other (poor choice of sampling design, date of survey) Think about why your variables are related. Causation is not the same thing as association! Is the relationship between your variables based on: Causation Confounding Common response Principles of Ethical Experiments Planned studies should be reviewed by a board to protect subjects from harm. All subjects must give their informed consent before data are collected. All individual data must be kept confidential. Only summaries can be made public. (Anonymity is not the same as confidentiality.)

2
VOCABULARY Population: The entire group of units or individuals about which we desire information. Sample: The part of the population selected to be measured or observed in order to gather data

for analysis. Census: An attempt to contact every individual in the entire population. Response Variable: Variable we are interested in studying. Unit: An individual person, animal or object upon which the response variable is measured. Units are called individuals when they refer to people. EXAMPLE A forester is interested in determining the total number of trees that are planted on tree farms in Montana. The forester believes the number of trees varies with the size of the tree farm. He divides all such farms into four classes depending on their size. From each class, he selects a sample of 15 farms. He counts and records the total number of trees for each of the selected tree farms. Unit: Population: Sample: Response variable: MAJOR IDEA We are interested in one or more variables associated with a population of units. Because it is impossible or too expensive to measure the variables of interest on all the units in the population, we only measure the variables on a subset or a sample of units. We use the sample to draw conclusions about the population. To be useful, however, the sample must represent the population. SAMPLING DESIGN Anectdotal evidence is information based on haphazardly selected individual cases which often come to our attention because they are striking in some way. Voluntary Response Sample: A sample which consists of people who choose themselves by responding to a general appeal. (Also called non-random or convenience sampling.) Random or Probability-Based Sampling: A sample that is selected in such a way that each unit in the population has a non-zero chance of being chosen. (SRS, Stratified Random Sample, Multistage Sample, Capture-Recapture)

3
Types of Random Sampling Designs Simple Random Sample (SRS) of size n: A sample that is selected from the population in such a way that every set of n units has an equal chance of being the selected sample. (We use SPSS or other computer package to select the sample.) Stratified Random Sample: The population is first divided into groups of similar units. A SRS is then selected from each of the groups. Multistage sample: A sample in which successively smaller groups within the population are selected in

stages. It is typically used when our population is so large that it is difficult or impossible to get a list of all of the units in the population. Here you start by splitting the population into groups and randomly selecting a number of the groups. You can split these selected groups again using another variable. Finally, when you have the number of units down to a manageable size, you select a SRS of units from each of the groups. Capture-Recapture Sample: A type of repeated SRS sampling that biologists use to estimate the size of animal populations. This type of sample may also be used by the government when estimating the number of households in an area. Take a SRS from the population and label them (or tag them). Later take a new SRS and find the percent of this sample that were in the original sample. Assume the proportion tagged in the second sample is equivalent to the proportion of the population who were tagged in the original sample. The population size can thus be estimated.

4
In order to make sure that a random sample is really random, you should use either a Table of Random Digits or a statistical computer application like SPSS to select the sample. You will learn how to use SPSS to take a random sample in lab on Friday. BIAS IN SAMPLING Sampling Bias occurs when the sample systematically favors certain parts of the population over others. Types of Sampling Bias Undercoverage Bias: This can occur when the sample systematically excludes a portion of the population. If the excluded portion systematically differs with respect to the response from those units that are available for sampling, sampling bias will be introduced into the study. Example If you wanted to take a survey of people in Lafayette, could you use the phone book to select your sample? Why not? Nonresponse Bias: This can occur when a selected unit either cannot be contacted or refuses to cooperate. If the non-respondent systematically differs with respect to the variable of interest compared to those who do respond, then sampling bias will be introduced into the study. Example If you were able to find phone numbers for everyone in your sample group, what might happen if you called each one and asked if they would take part in your survey? Response Bias: When the behavior of the respondent or the interviewer changes the sample result such that the results do not agree with the true population value. Examples of this include the respondent lying, poor

interviewing techniques, wording of questions, race or sex of interviewer influencing respondent, etc. Example where respondents might lie: Example where the wording of the question might be a problem: Example where race/sex of interviewer might result in biased answers: Example where the interviewer might influence the respondent: EXPERIMENT OR OBSERVATIONAL STUDY Observational Study: observe units or individuals and measure variables of interest but