Primary Data Collection: Observation

Prof. Rushen Chahal

Prof. Rushen Chahal

Nature of Observation Research

The systematic process of recording patterns of occurrences or behaviors without normally communicating with the people involved.
mmm..

The needed information must be either observable or inferable; The behavior should be repetitive, frequent, or in some manner predictable; The behavior must be relatively short in duration.
Prof. Rushen Chahal

Nature of Observation Research

Natural vs. Contrived: Is the setting made up by the researcher or are you observing a naturally occurring event? Open vs. Disguised: Does the subject know the purpose of the research? Structured vs. Unstructured: Is the researcher taking detailed notes or is the researcher making more general observations? Garbologist - Forensic Observation: Going through peoples garbage to analyze consumption patterns. Sometimes used as generic term for forensic observation the clues people leave behind.
Prof. Rushen Chahal

Nature of Observation Research

Mechanical: Eye movement recorders Pulpilometer; Voice pitch analysis; People monitors in TV; Traffic counters; Audits Qualitative & Quantitative People Reader / Meter; Physiological measurement; devices lie detectors Galvanic skin responses; Electroencephalograph; Rapid Analysis Measurement System (RAMS); Scanners - such as the grocery BehaviorScan. Ethnographic: The study of human behavior in its natural context, involving observation of behavior and physical setting.
Prof. Rushen Chahal

Information Resources Inc.

Nature of Observation Research

You see what people actually do - rather than what they say they do; Firsthand information is less prone to biases; The observational data can be executed quickly and relatively accurately; Electronic collection such as scanners is more efficient than manual counts; Clients can also observe their customers along with the researcher.

Only physical or behavior can be measured; Cant measure attitudes, beliefs, intentions, or feelings; Not always a good representation of the general population; Interpretation is somewhat subjective depending on observation type; Data analysis is generally more qualitative than quantitative; It can be expensive and time consuming if subjects not readily available; Data can be time sensitive making predictive analysis tricky.
Prof. Rushen Chahal

Mystery Shopping
People who pose as customers and shop at a companys own stores or those of its competitors to collect data about customer-employee interactions and to gather observational data; they might also compare prices, displays, and the like.

Click for More Information

Prof. Rushen Chahal

Mystery Shopping Levels

The mystery shopper calls/contacts the business with a script with questions and notes the experience.

The mystery shopper visits the business and makes a quick purchase with little or no customer-employee interaction.

Prof. Rushen Chahal

Mystery Shopping Levels

The mystery shopper visits the business for a more involved experience to include a scripted scenario to gauge the customer-employee interaction. Level III usually does not involve a major purchase - although detailed discussion about the product or services will likely occur.

The mystery shopper visits the business with great knowledge about the products. This knowledge can be used to test the employees. A purchase - and/or issues related to the purchases - might likely be involved.

Prof. Rushen Chahal

Conducting Mystery Shopping

Measuring employee training; Preparing for or monitoring new competition ; Comparison shopping; Recognizing good employees; To build a profile of the shoppers patters of behavior.

Audits: Examination and verification of product sales; One-Way-Mirror-Observation: Practice of watching behaviors and activities from behind a
one-way-mirror; ESOMAR - formerly the European Society for Opinion Research offers mystery shopping guidelines.
Click for Information

Understanding Behavior Through Observation

Prof. Rushen Chahal

Observation Research on the Internet

Predictive Consumer Intelligence:
Web site should mirror the customers buying process; Click sequence / patterns should enhance predictive capability.

Conversation Trackers:
Used to develop a cohesive message from thousands of conversation threads in newsgroups, chatrooms, listservs, message boards, etc.

Prof. Rushen Chahal

Primary Data Collection: Experimentation

Prof. Rushen Chahal

Prof. Rushen Chahal

What is an Experiment?
An Experiment: A research approach in which one variable is manipulated and the effect on another variable is observed. Key Variables: Independent: variables you control directly such as price, packaging, distribution, product features, etc.; Dependent: variables you do not directly control such as sales or customer satisfaction - (might control them by manipulating the independent variable); Treatment: the independent variable manipulated during and experiment to measure its effect on the dependent variable; Extraneous: Factors you do not control but have to live with such as the weather.
Prof. Rushen Chahal

Demonstrating Causation
Whoops!

Research designed to determine whether a change in one variable likely caused an observed change in another.
A causal relationships must demonstrate three things: Concomitant Variation; Appropriate Time Order of Occurrence; Elimination of Other Possible Causal Factors.

Prof. Rushen Chahal

Demonstrating Causation
Must Meet Three Criteria
Whoops!

1 . Concomitant Variation:
A statistical relationship between variables;

2. Appropriate Time Order of Occurrence:

Change in an independent variable occurred before an observed change in the dependent variable;

3. Elimination of Other Possible Causal Factors:

No other independent variables are causing the change
If you eliminate the impossible, whatever remains, however improbable, must be the truth. Sherlock Holmes Prof. Rushen Chahal

Experimental Setting

Experiments conducted in a controlled setting.

Tests conducted outside the laboratory in an actual environment, such as a marketplace.

Prof. Rushen Chahal

Experimental Validity

The extent to which competing explanations for the experimental results observed can be ruled-out.

The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times.
Prof. Rushen Chahal

Experimental Notation
X = Independent Variable: Indicates the exposure of an individual or a group to an experimental treatment. This variable is something the researcher can change and manipulate. It is hoped that the change in the independent variable will cause a change in the dependent variable.

O = Dependent Variable: Indicates a variable the researcher cannot change directly. It is hoped that changing the independent variable will cause changes in the dependent variable. Thus the dependent variable is dependent on what the researcher does with the independent variable.

Prof. Rushen Chahal

Extraneous Variables
History: Intervention, between the beginning and end of an experiment, of outside variables that might change the dependent variable. Maturation: Changes in subjects occurring during the experiment that are not related to the experiment but which might affect subjects response to the treatment factor. Instrument Variation: Changes in measurement instruments (e.g., interviews or observers) that might affect measurements. Selections Bias: Systematic differences between the test group and the control group due to a biased selection process.
Prof. Rushen Chahal

Extraneous Variables
Mortality: Loss of test units or subjects during the course of an experiment, which might result in a nonrepresentativeness.

Testing Effect: An effect that is a by-product of the research process itself.

Regression to the Mean: Tendency of subjects with extreme behavior to move toward the average for that behavior during the course of the experiment.

Things like the weather, government regulations, competitors actions

Prof. Rushen Chahal

Controlling Extraneous Variables

Randomization:
The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics.

Physical Control:
Holding constant the value or level of extraneous variables throughout the course of an experiment.

Design Control:
Use of experimental design to control extraneous causal factors.

Statistical Control:
Adjusting for the effects of extraneous variables by statistically adjusting the value or the dependant variable for each treatment condition.
Prof. Rushen Chahal

Experimental Design, Treatment, and Effects

Experimental Design:
A test in which the researcher has control over and manipulates one or more independent variables.

Treatment Variable:
The independent variable that is manipulated in an experiment.

Experimental Effect:
The effect of the treatment variable on the dependent variable.

Prof. Rushen Chahal

Experimental Design, Treatment, and Effects

High Cost:
Is the research affordable? Will the research be beneficial & help solve problems? Has a cost & benefit analysis been done?

Security Issues:
Particularly critical with field experiments. The competition might be tipped-off. Are the data and findings secure?

Process Contamination:
People who unwittingly get caught into the survey. Outside factors unnaturally affecting the experiment. Participants who intentionally try to skew the results.
Prof. Rushen Chahal

Selected Experimental Designs

Pre-Experimental Design:
Designs that offer little or no control over extraneous factors.

One-Shot Case Study One-Group Pretest-Posttest Static-Group Comparison

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

One Shot Case Study

X O1

Change the independent variable, then measure the change in the dependent variable to see if there was in fact a change in the dependent variable that the researcher might conclude resulted from the change in the independent variable.

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

One-Group Pretest-Posttest

O1X O2

Same as One-Shot except measure the dependent variable before the change in the independent variables. The researcher is establishing a benchmark from which to gauge the change.

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

Static-Group Comparison

X O1 O2

It uses an experimental and a control, but subjects or test units are not randomly assigned to the two groups and no premeasurements are taken.

Prof. Rushen Chahal

True Experimental Designs

True Experimental Design:
Research using an experimental group and a control group, to which test units are randomly assigned.

Before and After With Control Group Solomon Four Group Design After Only With Control Group

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

Before and After With Control Group

O1X O 2 O3 O4

True experimental design that involves random assignment of subjects or test units to experimental and control groups and pre- and post -measurements of both groups.

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

Solomon Four-Group Design

Research in which two experimental groups and two control groups are used to control for all extraneous variables.

Experimental Group 1

Control Group 1 Experimental Group 2 Control Group 2

O1X O 2 O 3 O4 X O5 O6

Prof. Rushen Chahal

Selected Experimental Designs

O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable

After Only With Control Group

Subjects in the experiment are randomly assigned to experiment and control groups respectively. No pre measurements of the dependent variable are taken.

Experimental Group

X O1 O2

Control Group

Prof. Rushen Chahal

Quasi-Experiments

Studies in which the researcher lacks complete control over the scheduling of treatments or must assign respondents to treatments in a nonrandom manner.

Prof. Rushen Chahal

Quasi-Experiments

Research in which repeated measurement of an effect interrupts previous data patterns.

Interrupted time-series design with a control group.

Prof. Rushen Chahal

Test Markets

Testing of new product/service, or some element of the marketing mix, using an experimental or quasi experimental design.

Cities as Test Markets

Advertising expenses; Point-of-purchase materials; Coupons and sampling; Travel and set-up expenses; Need for customized research; Possible diversion of sales from your other products; Potentially bad press / public reaction if experiment fails; Letting competitors know what your company is doing; Falsely thinking the sample results are always representative of the population.
Prof. Rushen Chahal

Six Steps in a Test Market Study

1. Define the Objective: What do you hope to learn? What are the characteristics of the people/products of interest? 2. Select a Basic Approach: Simulated, controlled, or standard test? 3. Develop Detailed Test Procedures: How will you execute the study? Who will be involved? How long will it take and how much can you spend?

Prof. Rushen Chahal

Six Steps in a Test Market Study

4. Select the Test Market: Market should not be over tested; Should have little media spillover; Demographics should be similar to your target population; Market should be large enough to provide useful results; Distribution and other patterns should be similar to the nation. 5. Execute The Plan: How long should the test run? Who should execute it? 6. Analyze the results: Use qualitative and quantitative techniques when possible.

Prof. Rushen Chahal