Google Answers Logo
View Question
 
Q: Most appropriate statistical test? ( Answered 2 out of 5 stars,   1 Comment )
Question  
Subject: Most appropriate statistical test?
Category: Business and Money > Finance
Asked by: donson88-ga
List Price: $100.00
Posted: 20 Nov 2006 09:38 PST
Expires: 20 Dec 2006 09:38 PST
Question ID: 784245
I am currently working on a project for a client to find out the
effect that a a print ad campaign has on its sales. Basically, this
client has two main markets, one in Baltimore and one in St Peterburg.
In Baltimore, client has hired an ad firm to run print ads in the free
local newspapers. No other advertising is otherwise run in the 2
markets. This campaign was started 2 years ago in Baltimore.

The data that I have available to me from client are (from 2000-2005)
the total sales in the 2 markets and the per capita income level of
Baltimore. What sort of statistical tests should I use? Differences in
differences?

I'm thinking that sales should be affected by both advertising and per
capita income, giving a regressed equation of Sales = ? + ?Ads +
?Income + e where ? is the intercept term, Ads is =0 if there is no
campaign and =1 if there is a campaign, Income is the income per
capita and e is the error term.

Please advise me accordingly

Request for Question Clarification by omnivorous-ga on 20 Nov 2006 11:23 PST
Donson88 -

You're close here to the measure: you'd like to compare sales in the
two markets with H(0) being St. Petersburg, FL and H(1) being
Baltimore.  They'd be comparable -- except for income variations in
the markets, which can be adjusted assuming that you have:
* sales (units)
* sales (dollars)

However, conversion rates may be different in the two markets due to
local management or perhaps even structural differences (tighter
credit conditions; legal differences putting impediments in the way;
different market segments).  Do you have any indication of call
activity or lead generation?  Without this data, it will be part of e,
the error term -- but an extremely valuable part of the analysis.

Best regards,

Omnivorous-GA

Clarification of Question by donson88-ga on 20 Nov 2006 11:33 PST
Hi Omni,

Here's the clarification that you need.

Currently, we don't have the data or measure of credit availability or
legal condition. My project team is currently in the exploratory stage
with the client and we just need to get a rough idea of the effect
that advertising has. I suppose for the time being, it is acceptable
for all the other factors excluding advertising and income to be
grouped in the error term e.

Also, the pricing that my client employs is the same in the two
markets (ie no effect on sales due to changes in price) and so we have
the $ total of the sales.

Hope that's enough info.

Clarification of Question by donson88-ga on 20 Nov 2006 11:38 PST
Also, if the Income data is not crucial, it'll be ok to leave it out
of the regresson.
Answer  
Subject: Re: Most appropriate statistical test?
Answered By: omnivorous-ga on 20 Nov 2006 16:55 PST
Rated:2 out of 5 stars
 
Donson88 ?

You?d be seeking to use tests that give you the largest number of
samples to differentiate between H0 (no advertising) and H1 (cases
with advertising).  You have the following data sets:

Baltimore H0, 2000
St. Petersburg H0, 2000

Baltimore H0, 2001
St. Petersburg H0, 2001

Baltimore H0, 2002
St. Petersburg H0, 2002

Baltimore H0, 2003
St. Petersburg H0, 2003

Baltimore H1, 2004
St. Petersburg H0, 2004

Baltimore H1, 2005
St. Petersburg H0, 2005


LOOKING AT YOUR SALES DATA
============================

Despite our attempts at clarification, it is not entirely clear what
you have for sales.  If you have sales in units and/or dollars per
sale, you potentially have many data points, allowing tighter
measurement of statistical significance because there are more
?degrees of freedom?.

If you do NOT have detailed sales data, the same tests can be applied
for gross revenues ? but with fewer degrees of freedom.  This is only
logical because a few large sales might be skewing the total revenues
database.  For example, if we?re trying to measure sales of commercial
apartment buildings and each of the two markets has only a half-dozen
or dozen sales ? but they represent hundreds of millions of dollars.

Online Statistics
?Differences between Two Means (Independent Groups),? (undated)
http://onlinestatbook.com/chapter10/difference_means.html

TEST 1: The data set from 2000-2003 should allow some judgment on the
relative size of the Baltimore and St. Petersburg markets, potentially
with a large sample size in N(Baltimore) or N(St. Pete).

Let us assume that the results are different in those four years: at
this point you?ll be seeking a correlation or cause for the
difference.  It is at this point that you?d be looking for the closest
relationship for:

Sales = ? + ?(FACTOR) + e

 
WHAT FACTOR IS CRITICAL?
===========================

In trying to estimate whether population, per capital income or some
other factor is important, it is important to understand the
underlying population, as Peter FitzRoy notes in the ?Advertising?
section of his book, ?Analytical Methods for Marketing Management.?

If we?re dealing with an impulse purchase or a minor consumable,
perhaps it is better to look at the total population of the two
cities.  Thus, for shampoo sales or newspaper sales, population might
provide the best FACTOR.

If it is a product aimed at retired people, perhaps the best statistic
is the number of people above age 65 in each market.  So, for denture
cleaners or supplemental Medicare plans we can look to test that data
as a FACTOR.  Census numbers for 2005 show that the over-65 population
of St. Petersburg is 50% larger than for the Baltimore metro area.

If the product being sold is being consumed by only upper income
groups, per capita income may be the easiest FACTOR to use.  Boat
sales and possibly even mortgage closings might all be very responsive
to this measure.

Luckily, the U.S. Census has started to speed the analysis of major
metropolitan areas by conducting annual surveys, providing much more
rapid demographic data than the 10-year census does.  Note that
Baltimore data is under Washington-Baltimore before 2005 but is
readily accessible.

In addition, the aggregation of St. Petersburg with the larger Tampa
market may cause some measurement problems for you:

U.S. Census Bureau
American Community Survey home
http://www.census.gov/acs/www/

Whatever predictive FACTOR you?re using, it should be one of the
elements of the 2004 and 2005 analysis, especially because
traditionally the St. Petersburg population sizes have been growing
more rapidly than the Baltimore SMSA.  Baltimore?s population grew
5.1% between 1995 and 2005, while Tampa-St. Petersburg grew by 20.4%
during the same 10 years.


IMPACT OF ADVERTISING
======================

TEST2: You?ve already drawn some conclusions about Baltimore and St.
Petersburg markets, results that will PARTIALLY account for
differences in 2004 and 2005 sales results.  Now it is time to add in
advertising to see what impact it is having:

Sales = ? + ?(FACTOR) + ?Ads + e

Here the H0 hypothesis is that the means are different -- ?0 for St.
Petersburg and ?1 for the Baltimore case.  The ? value should be
carefully considered: it might be the NUMBER of ads (if uniform); the
total DISPLAY SPACE; or possibly even the AMOUNT spent on advertising.

Note that there are reasons for excluding the ad firm?s creative fees
if they are upfront costs unrelated to starting a campaign and NOT
related to number of exposures. But there are also reasons to include
them if a campaign changes messages often ? or if you?re trying to do
a profitability analysis.

You should be able to tell quickly now whether the advertising is
having a significant impact by measuring the difference between ?0 and
?1 with a T-test.

And you should expect the advertising analysis to yield some
additional information:
?	assuming that advertising is roughly equal, year 2 advertising
should show an improvement in effectiveness due to diffusion effects
and lags in advertising response, among other impacts.  If it is not,
the strategy and tactics bear re-examination.
?	the analysis of the impact of advertising should allow an analysis
of the profitability of advertising by examining the increased
spending vs. increased profitability.  In one well-known marketing
analysis of retail done by Doyle and Fenwick, ?Planning and Estimation
in Advertising,? in the Journal of Marketing Research (1975), they
found that retailers benefited from increased sales as advertising
expenditures rose ? but that profits actually declined for the top 25%
of advertisers.

Google search strategy:
sales advertising "statistical test"
?difference between means?
advertising diffusion effect

There are likely to be aspects of this analysis that appear unclear. 
Please don?t hesitate to ask for a clarification request before rating
the answer.

Best regards,

Omnivorous-GA

Request for Answer Clarification by donson88-ga on 20 Nov 2006 18:14 PST
Thanks for your answer. I need some clarifications.

The sales data that I have is number of units sold per city in each
year, so I only have 12 data points in total (for the two cities over
6 years). Clearly, the different population sizes for the two cities
will affect the number of units sold, so would it work to use
Sales-Per-1000-Residents work in lieu of just sales?

Secondly, the amount spent on advertising in Baltimore is the same
over the two years. Given this fact, does your second regression
equation Sales = ? + ?(FACTOR) + ?Ads + e work by taking Ads to be a
dummy variable (ie =0 in St Petersburg & =1 in Baltimore?)

Request for Answer Clarification by donson88-ga on 20 Nov 2006 18:24 PST
Also, I have MicrosoftExcel. How do I run this regression? Which
regression would I use?

Will tip generously for a speedy response. Thanks again.

Clarification of Answer by omnivorous-ga on 21 Nov 2006 05:48 PST
Donson88 --

> Would it work to use Sales-Per-1000-Residents work in lieu of just sales?

Yes.  You always want the most-representative segment of the
population, but as mentioned in the original you're seeking a ?1 tied
to St. Pete and a ?2 for Baltimore so that you can adjust to each
city's change in the relevant population.

> does your second regression equation Sales = ? + ?(FACTOR) + ?Ads +
e work by > taking Ads to be a dummy variable

Yes.  Although you can run this as a simple difference of means test
using your calculated ? values from 2000-2003.

> How do I run this regression?

If you have the Data Analysis tools installed in Excel, you can run a
regression under Tools/Data Analysis.  I might use the @SLOPE to look
at correlation between St. Petersburg and Baltimore for 2000-2003 --
though you can also use FORECAST or TREND, as any will give you a
correlation for the two markets.

Once you've derived the beta for Sales-Per-1000-Residents, run a
predictive model for 2004 and 2005 in Baltimore to see if "?Ads" makes
any difference in sales.  For that the T-statistics will work fine.

Best regards,

Omnivorous-GA
donson88-ga rated this answer:2 out of 5 stars
Reasonable answer

Comments  
Subject: Re: Most appropriate statistical test?
From: ubiquity-ga on 21 Nov 2006 09:09 PST
 
Your methodology is not inherentlyu flawed, but your data is not
terribly useful without adding more peaces to the puzzle.  First you
must look at the growth of the industry (or sales in each city).  How
do you know the result was from ads and not simply the growth of the
market.  (Look at the number of the market as a whole ad that of your
competitors).

There could also be technologuical advances or other demands that
might cause an increase in sales in one city over another at a
particular poi8nt in time.

To be thorough, you must look at several factors.  That not to say
they all have to be in a single regression formula.  I am saying to do
right by your client you should review all available sources that my
be relevant.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy