Research and Experimental Design

Edward O. Garton, Jon S. Horne,

Jocelyn L. Aycrigg, and John T. Ratti
2012

Department of Fish and Wildlife Sciences, University of

Idaho, Moscow, Idaho

Chapter 1 In Silvy, Nova (ed.). Techniques For Wildlife Investigations

and Management. Seventh edition, Vol. I Wildlife Research Methods.
The Wildlife Society, Bethesda, MD.
 Crisis!
► There is a crisis in wildlife, fisheries, ecology and
conservation biology.
► Our practice of scientific method is broken.
► Our rigor in designing research has been lost in a rush
to investigate.
► We have failed to lay the proper foundation for
experiments, models and samples.
► Even worse, we have failed to confront our
understanding (theories) with data to improve and
correct them.
 Twelve-Step Program for
Recovering Naturaholics to Bring
Back Rigor
► 1. Identify problem and set objectives.
► 2. Conduct literature review and contact experts.
► 3. Collect preliminary observations and apply exploratory
data analysis to them.
► 4. Formulate a theory (conceptual model or research
hypothesis).
► 5. Formulate multiple predictions from conceptual model
as testable hypotheses or alternate models.
► 6. Design tests/experiments/models to test the theory.
 Twelve-Step Program (cont.)
► 7. Do a pilot study (techniques, variances, costs).
► 8. Revise proposal with a statistician based on results of
pilot study and reviews by experts.
► 9. Do experiment, collect observations and/or build model.
► 10. Confront theories with data (analyze it).
► 11. Evaluate, interpret and draw conclusions.
► 12. Use results and publish them for all to use and build
on.
Scientific Method (Conceptual)
 Overview: Formal Sequence
of 18 Steps
Step Example
1. Identify the research problem. 1. What are the influences of
environmental factors, such as
wildfire and winter severity, on
the carrying capacity of elk
winter range?
2. Conduct literature review of
relevant topics. 2. Excellent earlier work by
Houston 1982, Merrill & Boyce
1991, …
3. Identify broad and basic research
objectives. 3. Determine temporal and spatial
differences in food…

See additional steps and remainder of example in Box 1.1 of book

chapter.
 Initial Steps
► Problem Identification
 Applied research
 Basic research
► Literature Review
 Professional Society Publications:
►TWS, AFS, ESA, SCB, SAF, SRM, others

 Google Scholar, Web of Science, etc.

 Use Google, Yahoo and other general search engines
cautiously and critically
 Initial Steps: Defining Populations
(Biological, Political, and Research)
► Delimiting populations
 Defines potential applications (inference
space)
 Required for sampling
 Equally important for experiments (esp. field
experiments) and models
 Hierarchically Structured
Biological Units:
► Individual
► Deme

► Population
►Metapopulation
►Subspecies
►Species
 Deme
► “A group of individuals where breeding is random”
(Emlen 1984).
► “A panmictic population” (Ehrlich and Holm 1963)

► Identificationof demes and other groupings of

individuals should be based on demography, movement,
genetics and geography of individuals and habitats.
 Deme - Definition
► Thesmallest grouping of individuals
approximating random breeding, within
constraints of breeding system, where it is
reasonable to estimate birth, death, immigration
and emigration rates.
 Deme
► Genetics: Random breeding within constraints of social
system.
► Demography: Smallest grouping where its feasible to
estimate birth, death, immigration and emigration rates.
► Movement: Restricted to home ranges in key seasons.
► Geography: Continuous distribution of individuals
within one patch of habitat or a closely spaced set of
habitat patches.
 Population
►A collection of demes or individuals at one point
in time, typically the breeding season, with
strong connections demographically (very high
correlations in vital rates), geographically (close
proximity), genetically (Manel et al. 2005) and
through frequent dispersal.
 Population
► Movement: High rates of dispersal between
adjacent demes.
► Genetics: Very closely related genetically.
► Geography: A collection of closely-spaced
occupied patches of habitat without great
expanses of non-habitat intervening between
them.
► Demography: Very high correlations between
demes and small groups of individuals.
 Metapopulation
► A collection of populations sufficiently close together that
dispersing individuals from source populations occasionally
colonize empty habitat resulting from local population extinction
(Levins 1969).
► Populations within a single metapopulation may show low or
high correlations in demographic rates, but the low rates of
dispersal are still high enough to maintain substantial genetic
similarity.
► Numerous types of metapopulations have been described from
source-sink to non-equilibrium to Levins’ classic (Harrison and
Taylor 1997).
 Metapopulation
► Movement: Probability of dispersal between
populations is low but colonization occurs.
► Demography: Possible low correlations in rates
produces high independence whereas demographics of
“source” populations may drive demography of “sink”
populations.
► Genetics: Genetic differentiation occurs between
populations but not enough to become subspecies.
► Geography: Substantial areas of non-habitat may
separate populations and patches of suitable but
unoccupied habitat.
Biological, Political, and Research Populations
 Statistical/Research Population = Sampling
Universe = Sampling Frame
► Entireset of units (i.e., elements, potential
observations) that exist and from which we want to
make inference.

► Even though we want to make statements (i.e.,

estimates) for animal or plant populations we often
recast this population spatially:
 i.e., statistical population = all the spatial units
occupied by our biological population
 Statistical Population
► GIS a key tool for handling sampling frame
because we often lay it out spatially.
► We use sampling frame to define statistical
population to which we apply our estimates
► We also draw our samples randomly from the
sampling frame.
► But note that randomly doesn’t mean stupidly –
we can draw them cleverly while we draw them
randomly.
 Sampling or Experimental Units
► Non-overlapping collections of elements of
statistical population that cover entire statistical
population.
► Note that these sampling units may be
individuals, plots, spatial polygons, or
collections of elements.
Initial Steps
► Define Population(s)
► Preliminary Data Collection
 Get our feet wet and try out ideas
► Exploratory Data Analysis
► Search literature for others’ ideas
► Put on thinking cap
 Theory, Models, Predictions and Hypotheses
Conceptual Model of Waterfowl
Population Dynamics
 Components of Theory (after
Pickett et al. 2007)
► Domain: Scope in space, time and phenomena
addressed by a theory.
► Assumptions: Conditions needed to build
theory.
► Concepts: Labeled regularities in phenomena.
► Definitions: Conventions and descriptions
necessary for the theory to work with clarity
 Components of Theory
► Confirmed generalizations: Condensations and
abstractions from a body of facts that have been
tested or systematically observed.
► Laws or principles: Conditional statements of
relationship or causation, statements of identity,
or statements of process that hold within a
domain.
 Components of Theory
► Models: Conceptual constructs that represent or
simplify the structure and interactions in the
material world.
 Scientific models can project consequences of
ideas
 Statistical models draw inferences and
discriminate between competing ideas based on
limited observations
 Components of Theory
► Framework: Nested causal or logical structure of
a theory.
► Translation: Procedures and concepts needed to
move from the abstractions of a theory to the
specifics of applications or test or vice versa.
► Hypotheses: Testable statements derived from or
representing various components of theory.
Study Designs for Investigation Process
 Experiments
Strengths and weaknesses of different types of experiments (modified from Diamond 1986).
__________________________________________________________
Experiment Type
__________________________________________________________
Laboratory Field Natural
_____________________________
Control of independent variablesa Highest Medium Low
Ease of inference High Medium Low
Potential scale (time and space) Lowest Medium Highest
Scope (range of manipulations) Lowest Medium High
Realism Low High Highest
Generality Low Medium High
__________________________________________________________
aActive regulation and/or site matching.

See Table 1.1 in book chapter.

 Experimental Design
► Focus of statistics programs, especially in
agricultural regions
► Dependent experimental units
► Fixed, random, mixed, and nested effects
► Controls
► Replication
► Determining required sample size
 Checklist for Experimental Design
► 1. What is hypothesis to be tested?
► 2. What is response or dependent variable(s) and
how should it be measured?
► 3. What is independent or treatment variable(s)
and what levels of variable(s) should we test?
► 4. To which population do we want to make
inferences?
 Checklist for Experimental Design
► 5. What will be our experimental unit?
► 6. Which experimental design is best?
► 7. How large should the sample size be?
► 8. Have you consulted a statistician and received
peer review on your design?
 Modeling
► Powerful alternative for:
 solutions to pressing problems
 selecting best of alternatives
 effects from multiple simultaneous causes
 evaluating population viability
► Goal: build simplest scientific model with ability
to produce useable predictions
► Statistical models used for estimation,
hypothesis testing, and comparison
 Scientific Modeling Strategies
Description Simple Complex
Quantification Conceptual (verbal)------------Quantitative
Theoretical General-----------------Complex Simulation
Relationships Linear------------------------------Non-linear
Variability Deterministic----------------------Stochastic
Time Scale Time-specific------------------------Dynamic
Mathematical Difference ------------------------Differential
Formulation Equations Equations
Factors Single------------------------------Multifactor
Spatial Single Site---------------------------Multi-site
No. of Species Single Species------------------Multi-species
 Statistical Modeling Strategies
Description Simple Complex
Sampling Simple random----------Stratified, clustered, or
multi-stage
Hypothesis testing Fixed or -----------------Mixed fixed and random
random effects effects
Independence of Complete ----------------Dependence between
observations independence observations in space,
time, or both

Errors Single term---------------Separate process and

observation errors
 Steps to Build a Scientific Model
► Problem definition
► System identification
► Select model type
► Mathematical formulation
► Parameter estimation
► Model validation
► Model experimentation
Basic Concepts of Survey Sampling

►Survey sampling is a magic sword;

 but it’s a two-edged sword:
►Sharp edge slices through all kinds of problems producing
unbiased estimates even if some assumptions about
population are wrong
►Dulledge requires applying some type of random
sampling
 Designing a Survey
► 1. What is objective of survey?
► 2. What is best technique?
► 3. To which population will we make inferences?
► 4. What will we sample (sample unit)?
► 5. What is the size of population we are sampling
(N)?
► 6. Which sample design is best?
► 7. How large should sample be?
 Sampling Designs and Types of
Sampling Units or Experimental Units
 Key Terms and Concepts
►Precision: deviation of repeated samples from
each other.
►Bias: deviation of repeated samples from true
value.
►Accuracy: deviation of an individual sample
from true value.
Bias, Precision, and Accuracy
 Confronting Theories with Data
► Hypothesis testing (classic frequentist)
► Effect size and interval estimation
► Information theoretic model selection
► Regression and General Linear Models (GLM -
Fisherian)
► Bayesian Approaches
► Validating parametric and simulation models
Confronting Theories with Data
 Finishing
► Speculation and new hypotheses
► Publication will
 Correct errors
 Insure sound conclusions
 Facilitate defending
controversial practices
 Help personnel grow in skills
 Make a permanent contribution to wildlife
knowledge
 Common Problems to Avoid
►Procedural inconsistency
►Non-uniform treatments
►Pseudoreplication
►Insufficient sample size
 Adaptive Management: Connecting
Research and Management
► Specifymanagement goals formally
► Use predictive models to:
 Summarize existing knowledge of factors and processes
producing desired responses
 Forecast results of alternative management actions
 Select best management action to implement, treating it as an
experiment to evaluate
► Monitor what happens to:
 Ensure goals are met
 Correct relationships and processes assumed in management
 Adaptive Management: Connecting
Research and Management
► Alternatives:
 Structured Decision Making
 Scenario Planning
 Various forms of business decision-making
 Summary
► Carefully designed wildlife research will improve reliability of
knowledge base of wildlife management.
► Research biologists must rigorously apply scientific method and
make use of powerful techniques in survey sampling, experimental
design and information theory.
► We must move from observational studies to experimental studies,
replicated across space and time, that provide a more reliable basis
for interpretation and conclusions.
► Modeling is an effective tool to predict consequences of
management choices, especially when based on carefully designed
field studies, long-term monitoring, and management experiments
designed to increase understanding.
 Summary
► Wildlife biologists have tremendous responsibility
associated with management of animal species
experiencing:
 increasing environmental-degradation problems
 loss of habitat
 declining populations
► We must face these problems armed with knowledge
from quality scientific investigations.