View Question
Q: Most appropriate statistical test? ( Answered ,   1 Comment )
 Question
 Subject: Most appropriate statistical test? Category: Business and Money > Finance Asked by: donson88-ga List Price: \$100.00 Posted: 20 Nov 2006 09:38 PST Expires: 20 Dec 2006 09:38 PST Question ID: 784245
 ```I am currently working on a project for a client to find out the effect that a a print ad campaign has on its sales. Basically, this client has two main markets, one in Baltimore and one in St Peterburg. In Baltimore, client has hired an ad firm to run print ads in the free local newspapers. No other advertising is otherwise run in the 2 markets. This campaign was started 2 years ago in Baltimore. The data that I have available to me from client are (from 2000-2005) the total sales in the 2 markets and the per capita income level of Baltimore. What sort of statistical tests should I use? Differences in differences? I'm thinking that sales should be affected by both advertising and per capita income, giving a regressed equation of Sales = ? + ?Ads + ?Income + e where ? is the intercept term, Ads is =0 if there is no campaign and =1 if there is a campaign, Income is the income per capita and e is the error term. Please advise me accordingly``` Request for Question Clarification by omnivorous-ga on 20 Nov 2006 11:23 PST ```Donson88 - You're close here to the measure: you'd like to compare sales in the two markets with H(0) being St. Petersburg, FL and H(1) being Baltimore. They'd be comparable -- except for income variations in the markets, which can be adjusted assuming that you have: * sales (units) * sales (dollars) However, conversion rates may be different in the two markets due to local management or perhaps even structural differences (tighter credit conditions; legal differences putting impediments in the way; different market segments). Do you have any indication of call activity or lead generation? Without this data, it will be part of e, the error term -- but an extremely valuable part of the analysis. Best regards, Omnivorous-GA``` Clarification of Question by donson88-ga on 20 Nov 2006 11:33 PST ```Hi Omni, Here's the clarification that you need. Currently, we don't have the data or measure of credit availability or legal condition. My project team is currently in the exploratory stage with the client and we just need to get a rough idea of the effect that advertising has. I suppose for the time being, it is acceptable for all the other factors excluding advertising and income to be grouped in the error term e. Also, the pricing that my client employs is the same in the two markets (ie no effect on sales due to changes in price) and so we have the \$ total of the sales. Hope that's enough info.``` Clarification of Question by donson88-ga on 20 Nov 2006 11:38 PST ```Also, if the Income data is not crucial, it'll be ok to leave it out of the regresson.```
 ```Donson88 ? You?d be seeking to use tests that give you the largest number of samples to differentiate between H0 (no advertising) and H1 (cases with advertising). You have the following data sets: Baltimore H0, 2000 St. Petersburg H0, 2000 Baltimore H0, 2001 St. Petersburg H0, 2001 Baltimore H0, 2002 St. Petersburg H0, 2002 Baltimore H0, 2003 St. Petersburg H0, 2003 Baltimore H1, 2004 St. Petersburg H0, 2004 Baltimore H1, 2005 St. Petersburg H0, 2005 LOOKING AT YOUR SALES DATA ============================ Despite our attempts at clarification, it is not entirely clear what you have for sales. If you have sales in units and/or dollars per sale, you potentially have many data points, allowing tighter measurement of statistical significance because there are more ?degrees of freedom?. If you do NOT have detailed sales data, the same tests can be applied for gross revenues ? but with fewer degrees of freedom. This is only logical because a few large sales might be skewing the total revenues database. For example, if we?re trying to measure sales of commercial apartment buildings and each of the two markets has only a half-dozen or dozen sales ? but they represent hundreds of millions of dollars. Online Statistics ?Differences between Two Means (Independent Groups),? (undated) http://onlinestatbook.com/chapter10/difference_means.html TEST 1: The data set from 2000-2003 should allow some judgment on the relative size of the Baltimore and St. Petersburg markets, potentially with a large sample size in N(Baltimore) or N(St. Pete). Let us assume that the results are different in those four years: at this point you?ll be seeking a correlation or cause for the difference. It is at this point that you?d be looking for the closest relationship for: Sales = ? + ?(FACTOR) + e WHAT FACTOR IS CRITICAL? =========================== In trying to estimate whether population, per capital income or some other factor is important, it is important to understand the underlying population, as Peter FitzRoy notes in the ?Advertising? section of his book, ?Analytical Methods for Marketing Management.? If we?re dealing with an impulse purchase or a minor consumable, perhaps it is better to look at the total population of the two cities. Thus, for shampoo sales or newspaper sales, population might provide the best FACTOR. If it is a product aimed at retired people, perhaps the best statistic is the number of people above age 65 in each market. So, for denture cleaners or supplemental Medicare plans we can look to test that data as a FACTOR. Census numbers for 2005 show that the over-65 population of St. Petersburg is 50% larger than for the Baltimore metro area. If the product being sold is being consumed by only upper income groups, per capita income may be the easiest FACTOR to use. Boat sales and possibly even mortgage closings might all be very responsive to this measure. Luckily, the U.S. Census has started to speed the analysis of major metropolitan areas by conducting annual surveys, providing much more rapid demographic data than the 10-year census does. Note that Baltimore data is under Washington-Baltimore before 2005 but is readily accessible. In addition, the aggregation of St. Petersburg with the larger Tampa market may cause some measurement problems for you: U.S. Census Bureau American Community Survey home http://www.census.gov/acs/www/ Whatever predictive FACTOR you?re using, it should be one of the elements of the 2004 and 2005 analysis, especially because traditionally the St. Petersburg population sizes have been growing more rapidly than the Baltimore SMSA. Baltimore?s population grew 5.1% between 1995 and 2005, while Tampa-St. Petersburg grew by 20.4% during the same 10 years. IMPACT OF ADVERTISING ====================== TEST2: You?ve already drawn some conclusions about Baltimore and St. Petersburg markets, results that will PARTIALLY account for differences in 2004 and 2005 sales results. Now it is time to add in advertising to see what impact it is having: Sales = ? + ?(FACTOR) + ?Ads + e Here the H0 hypothesis is that the means are different -- ?0 for St. Petersburg and ?1 for the Baltimore case. The ? value should be carefully considered: it might be the NUMBER of ads (if uniform); the total DISPLAY SPACE; or possibly even the AMOUNT spent on advertising. Note that there are reasons for excluding the ad firm?s creative fees if they are upfront costs unrelated to starting a campaign and NOT related to number of exposures. But there are also reasons to include them if a campaign changes messages often ? or if you?re trying to do a profitability analysis. You should be able to tell quickly now whether the advertising is having a significant impact by measuring the difference between ?0 and ?1 with a T-test. And you should expect the advertising analysis to yield some additional information: ? assuming that advertising is roughly equal, year 2 advertising should show an improvement in effectiveness due to diffusion effects and lags in advertising response, among other impacts. If it is not, the strategy and tactics bear re-examination. ? the analysis of the impact of advertising should allow an analysis of the profitability of advertising by examining the increased spending vs. increased profitability. In one well-known marketing analysis of retail done by Doyle and Fenwick, ?Planning and Estimation in Advertising,? in the Journal of Marketing Research (1975), they found that retailers benefited from increased sales as advertising expenditures rose ? but that profits actually declined for the top 25% of advertisers. Google search strategy: sales advertising "statistical test" ?difference between means? advertising diffusion effect There are likely to be aspects of this analysis that appear unclear. Please don?t hesitate to ask for a clarification request before rating the answer. Best regards, Omnivorous-GA``` Request for Answer Clarification by donson88-ga on 20 Nov 2006 18:14 PST ```Thanks for your answer. I need some clarifications. The sales data that I have is number of units sold per city in each year, so I only have 12 data points in total (for the two cities over 6 years). Clearly, the different population sizes for the two cities will affect the number of units sold, so would it work to use Sales-Per-1000-Residents work in lieu of just sales? Secondly, the amount spent on advertising in Baltimore is the same over the two years. Given this fact, does your second regression equation Sales = ? + ?(FACTOR) + ?Ads + e work by taking Ads to be a dummy variable (ie =0 in St Petersburg & =1 in Baltimore?)``` Request for Answer Clarification by donson88-ga on 20 Nov 2006 18:24 PST ```Also, I have MicrosoftExcel. How do I run this regression? Which regression would I use? Will tip generously for a speedy response. Thanks again.``` Clarification of Answer by omnivorous-ga on 21 Nov 2006 05:48 PST ```Donson88 -- > Would it work to use Sales-Per-1000-Residents work in lieu of just sales? Yes. You always want the most-representative segment of the population, but as mentioned in the original you're seeking a ?1 tied to St. Pete and a ?2 for Baltimore so that you can adjust to each city's change in the relevant population. > does your second regression equation Sales = ? + ?(FACTOR) + ?Ads + e work by > taking Ads to be a dummy variable Yes. Although you can run this as a simple difference of means test using your calculated ? values from 2000-2003. > How do I run this regression? If you have the Data Analysis tools installed in Excel, you can run a regression under Tools/Data Analysis. I might use the @SLOPE to look at correlation between St. Petersburg and Baltimore for 2000-2003 -- though you can also use FORECAST or TREND, as any will give you a correlation for the two markets. Once you've derived the beta for Sales-Per-1000-Residents, run a predictive model for 2004 and 2005 in Baltimore to see if "?Ads" makes any difference in sales. For that the T-statistics will work fine. Best regards, Omnivorous-GA```
 donson88-ga rated this answer: `Reasonable answer`
 ```Your methodology is not inherentlyu flawed, but your data is not terribly useful without adding more peaces to the puzzle. First you must look at the growth of the industry (or sales in each city). How do you know the result was from ads and not simply the growth of the market. (Look at the number of the market as a whole ad that of your competitors). There could also be technologuical advances or other demands that might cause an increase in sales in one city over another at a particular poi8nt in time. To be thorough, you must look at several factors. That not to say they all have to be in a single regression formula. I am saying to do right by your client you should review all available sources that my be relevant.```