Marketing-Experiments-slides PDF

Marketing Analytics
Marketing Experiments Slides

These materials are for your personal use while participating in this course. Please do not share or distribute them.
Marketing Experiments
Video 1: Welcome to Week 4
Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Business Marketing Analytics course 1
Marketing Analytics
Introduction
• Experimentation is a key component of the resource
allocation portfolio that allows us to evaluate the
consequence of different marketing actions.
• In this module we will
 Understand some basic experimental designs
 Follow a case study of a firm that implemented experiments on TV
and the web
By the end of this module, you will be able to…
•Define correlation versus causation

•Design effective experiments to test marketing
hypotheses
•Conduct basic experiments to assess the
effectiveness of marketing efforts

Marketing Analytics
Video 2: Determining Cause and

Effect through Experiments
Correlation vs Causation
•Does skipping breakfast cause obesity?
 http://www.webmd.com/diet/news/20080303/eating-
breakfast-may-beat-teen-obesity
•Alternative explanations:
 Physical activity
 Lack of sleep

Marketing Analytics
“Half the money I spend on advertising is wasted;

the trouble is I don’t know which half”
John Wanamaker
Father of Modern Advertising
Marketing Return on Investment

•Why is it hard to measure the Return on
Marketing Spending?
 TV Advertisements
 Promotion
 Better Customer Targeting
•Basic Issue: Would you have achieved the same
sales increase without the increased advertising
spend?

Marketing Analytics
Video 3: Designing Basic

Experiments
What Establishes Causality?

• Change in marketing mix produces change in sales
 Increasing Advertising $ Increased Sales
• No sales increase when there is no change in the marketing mix
 No Increase in Advertising $ Same Sales
• Time Sequence
 Increased advertising $ today leads to higher sales tomorrow
• No other external factor
 When advertising was increased, one of the competitors left the market. So sales
increased because of lesser competition not because of increased advertising.

Marketing Analytics
Experiments – The Holy Grail!

One or more independent
variable(s) [Advertising $] are
manipulated to observe changes
in the dependent variable
[Sales or Brand awareness]
A Basic Experiment
Choose 1,000 How to assign
Customers customers to
Experiment and
Control Group?
Test Group (500) Control Group (500)
Exposed to advertisement highlighting Exposed to old advertisement
new packaging for one month for one month
Test Group Control Group

Total Sales = 1,200 units Total Sales = 1,000 units
Sales Lift
Test-Control = 200 units

Marketing Analytics
Video 4: Designing Before-

After Experiments
A Basic Experiment Randomization

can match test
and control
Choose 1,000 groups on all
Customers dimensions
simultaneously,
Test Group (500)
Exposed to advertisement highlighting
Control Group (500)
Exposed to old advertisement
given a sufficient
new packaging for one month for one month sample size

Sales Lift

Marketing Analytics
A Basic Experiment Match

experiment and
control groups on
Choose 1,000 known attributes
Customers (e.g., demographics)

Exposed to advertisement highlighting
new packaging for one month
Exposed to old advertisement
for one month
When would this
be necessary?
Sales Lift
Before – After Design
Choose 1,000 Customers

Exposed to advertisement highlighting new packaging for Exposed to old advertisement
one month for one month

Sales Lift

Marketing Analytics
Video 5: Designing Full

Factorial Web Experiments
Before – After Design

Choose 1,000 Customers
Old Advertisement Old Advertisement
Exposed to new advertisement

Old Advertisement
highlighting new packaging for one month
Total Sales = 1,200 units Total Sales = 1,000
Sales Lift
Test-Control = (1,200 – 1,000) – (1,100 – 1,000)
= 100 units

Marketing Analytics
Web Experiments – Full Factorial Design

Price
Advertisement Copy $1.59 $1.89* $2.15
“Lasts Longer” $1,315 $1,112 $1,206
“Tastes Better” $957 $1,030 $1,500
“Good for You”* $930 $820 $770
* Current conditions, so can be considered controls.

Price
“Lasts Longer” $1,315 $1,112 $1,206
“Tastes Better” $957 $1,030 $1,500
“Good for You”* $930 $820 $770

Marketing Analytics

Price
“Lasts Longer” $1,315 $1,112 $1,206
“Tastes Better” $957 $1,030 $1,500
“Good for You”* $930 $820 $770

Price
“Lasts Longer” $1,315 $1,112 $1,206
“Tastes Better” $957 $1,030 $1,500
“Good for You”* $930 $820 $770

Marketing Analytics

Price
“Lasts Longer” $1,315 $1,112 $1,206
“Tastes Better” $957 $1,030 $1,500
“Good for You”* $930 $820 $770
Video 6: Designing an
Experiment: Etch A Sketch

Marketing Analytics
Photo Credit: Etcha (Own work) [CC BY-SA 3.0 Photo Credit: By Shikhar Sharma (Own work) [CC BY-SA 3.0
(http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia
(http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons Commons
Sample Retailer Weekly Unit Sales (Etch A Sketch)
14000
12000
10000
Sales (units)
8000
6000
4000
2000
0
1-Feb 1-Mar 1-Apr 1-May 1-Jun 1-Jul 1-Aug 1-Sep 1-Oct 1-Nov 1-Dec 1-Jan

Marketing Analytics
Video 7: Analyzing an
Experiment: Etch A Sketch
Etch A Sketch Test Results

Etch A Sketch Doodle
Test Product Control Product
Number Cincinnati Control* Cincinnati Cincinnati Control Cincinnati
of Weeks Units Units Shares (%) Units Units Share
Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%
* Control Cities included (a) Charleston, South Carolina; (b) Cleveland, Ohio; (c) Indianapolis, Indiana; and (d)
Pittsburg, Pennsylvania

Marketing Analytics

Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%

Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%

Marketing Analytics

Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%

Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%

Marketing Analytics

Pre Test
5 Dec 2005 – 26 Nov 2006
12 162 1526 9.6 1517 6742 18.4
Test
27 Nov 2006 – 16 Dec 2006
3 240 1598 13.1 816 3780 17.7
Lift 136.1 96.7
Net Lift 39.4%

Retail Price 10
Retail Margin 36%
Manufacturer Selling Price 6.4
Manufacturer Contribution Margin % 58%
Manufacturer Contribution Margin $ 3.71
National Budget 5,000,000

Units Break Even 1,346,983
Base Units 3,100,000
Base Units Test Period 1,085,000
Break Even Lift % of Base 224%

Marketing Analytics

Retail Price 10
Retail Margin 36%


Retail Price 10
Retail Margin 36%


Marketing Analytics
Video 8: Analyzing a Market

Experiment: Betty Spaghetty
Betty Test Results
Arizona California
Color Crazy Go Go Glam Color Crazy Go Go Glam
Total/Store/Week
1.8 2.2 0.3 1.2
17 Jun – 17 Jul 2007
Lift 267%

Marketing Analytics
Betty Test Results
Arizona California
Total/Store/Week
1.8 2.2 0.3 1.2
17 Jun – 17 Jul 2007
Lift 267%
Betty Test Results
Arizona California
Total/Store/Week
1.8 2.2 0.3 1.2
17 Jun – 17 Jul 2007
Lift 267%

Marketing Analytics
Betty Test Results
Arizona California
Total/Store/Week
1.8 2.2 0.3 1.2
17 Jun – 17 Jul 2007
Lift 267%
Betty Test Results

Ad Budget 3,000,000
Retail Selling Price 15
Retail Margin % 36%
Manufacturer Suggested Price 9.6
Break Even Units 538,793

Marketing Analytics
Video 9: Projecting Lift
Arizona Test

Marketing Analytics
Arizona Chain Sales
Arizona Test
National Chain Sales
Arizona Chain Sales
Arizona Test

Marketing Analytics
All Retail
National Chain
Sales
Arizona Chain
Sales
Arizona Test
Selling Season
All Retail
National Chain Sales
Arizona Chain Sales
Arizona Test

Marketing Analytics
Video 10: Calculating

Projected Lift: Betty
Spaghetty
Betty Test Results

California Test Units 142
Test % of California Sales 10%
Total California Units 1420
California % of National Sales 12%
National Retailer Sales 11,833
Retailer Share 25%

Marketing Analytics
Betty Test Results

National Units 47,333
Test % of Annual Sales 5.50%
Annual Sales 860,606
Holiday % of Annual 45%
Holiday Units Without Ads 387,273
Lift From Ads 267%
Units After Ads 1,034,018
Betty Test Results

Ad budget 3,000,000
Retail selling price 15
Retail margin % 36%
Manufacturer selling price 9.6
Manufacturer contribution margin % 58%
Manufacturer contribution margin $ 5.568
Break even units 538,793

Marketing Analytics
Video 11: Pitfalls of Marketing

Experiments: Betty Spaghetty
2008 Holiday Season

•Betty Spaghetty was supported with a $2M
advertising campaign

Marketing Analytics
2008 Holiday Season

•Betty Spaghetty was supported with a $2M
advertising campaign
•Sales not as expected
 Upstaged by Hannah Montana
Video 12: Maximizing

Effectiveness: Nanoblocks

Marketing Analytics
The Nanoblock Amazon Goldbox Experiment –

March 2012
Nano Eiffel Tower – Amazon Goldbox

Promotion
Units Sold
Lift on
units sold
Sales Jan – Feb Mar May (promotion vs.
Price 2012 2012 2012 pre-promotion)
Nanoblock Eiffel Tower $19.99 274 686 219 501%
Nanoblock Taj Mahal $19.99 308 163 132 106%
Nanoblock Castle Neuschwanstein $19.99 244 184 146 151%
Classic Etch a Sketch $12.99 344 352 399 205%

Marketing Analytics
Gold Box Economics
% Lift Incremental Incremental

From Feb Units Revenue
Eiffel Regular 182% 249 $4,978
Eiffel Sale 300 $1,797
Etch A Sketch 105%
Goldbox Promotions Provide Dividends in

Search Results

Marketing Analytics

Marketing Analytics

Marketing Analytics
Video 13: Marketing

Experiments Takeaways

Marketing Analytics
Conclusion
• Experiments assess cause and effect
• Pay attention to:
 Design
 Gap between test results and field implementation
 Difference between test and campaign contexts
• Web experiments are cheaper and faster
 Costs of experiments can be variable rather than fixed
• Experiments provide forecasts of expected ROI
 This can help with determining campaign budgets
Now that we’re done, you should be able to…
•Define correlation versus causation

•Design effective experiments to test marketing
hypotheses
•Conduct basic experiments to assess the
effectiveness of marketing efforts


Marketing-Experiments-slides PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Marketing-Experiments-slides PDF

Загружено:

Авторское право:

Доступные форматы

Marketing Analytics

Marketing Experiments Slides

Video 1: Welcome to Week 4

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

By the end of this module, you will be able to…

•Define correlation versus causation

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Video 2: Determining Cause and

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

“Half the money I spend on advertising is wasted;

Marketing Return on Investment

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Video 3: Designing Basic

What Establishes Causality?

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Experiments – The Holy Grail!

Test Group Control Group

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Video 4: Designing Before-

A Basic Experiment Randomization

Test Group Control Group

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

A Basic Experiment Match

Test Group (500) Control Group (500)

Before – After Design

Choose 1,000 Customers

Test Group (500) Control Group (500)

Test Group Control Group

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Video 5: Designing Full

Before – After Design

Total Sales = 1,100 units Total Sales = 1,000 units

Exposed to new advertisement

Total Sales = 1,200 units Total Sales = 1,000

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Web Experiments – Full Factorial Design

* Current conditions, so can be considered controls.

Web Experiments – Full Factorial Design

* Current conditions, so can be considered controls.

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Web Experiments – Full Factorial Design

* Current conditions, so can be considered controls.

Web Experiments – Full Factorial Design

* Current conditions, so can be considered controls.

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Web Experiments – Full Factorial Design

* Current conditions, so can be considered controls.

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Sample Retailer Weekly Unit Sales (Etch A Sketch)

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Etch A Sketch Test Results

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Etch A Sketch Test Results

Etch A Sketch Test Results

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Etch A Sketch Test Results

Etch A Sketch Test Results

Developed by Raj Venkatesan for the University of Virginia’s Darden School of

Etch A Sketch Test Results

Etch A Sketch Test Results

National Budget 5,000,000

Break Even Lift % of Base 224%

Developed by Raj Venkatesan for the University of Virginia’s Darden School of