My company calculates absolute percent error the following way:
(ABS(FCST-SALES)/FCST)	
There are two absolute error calculations.  One is based on the
customers orignal forecast vs. what they bought and the other is based
on a system created forecast (linear demand based on history) vs. what
they bought.  At the end of each month the system spits out the two
errors.  My issue is if I only have the percent error for each of the
forecasts how would I go about calculating percent difference....I
thought it was just a simple subtraction (i.e. one minus the other)
but someone else is actually calculating percent difference...who is
right and why?

---

Request for Question Clarification by efn-ga on 14 Sep 2004 20:24 PDT

Hi massradius-ga,

For what pair or pairs of numbers do you want the percent difference? 
Are you looking for two percent differences where the inputs are the
same as the inputs to the absolute error calculations, that is, a
customer forecast vs. actual sales percent difference and separately,
a system forecast vs. actual sales percent difference?

Also, what definition of percent difference do you want to use?  One
common definition is the difference between two numbers expressed as a
percentage of one of the numbers.  But there is another definition
used in science, "the absolute value of the difference over the mean
times 100," sometimes known as the "relative percent difference."

Source:

http://www.phy.ilstu.edu/~wenning/ptefiles/302content/student_lab_hdbk/Percent%20Difference%20Error.pdf

--efn

---

Clarification of Question by massradius-ga on 15 Sep 2004 06:44 PDT

Hi massradius-ga,

For what pair or pairs of numbers do you want the percent difference? 
Are you looking for two percent differences where the inputs are the
same as the inputs to the absolute error calculations, that is, a
customer forecast vs. actual sales percent difference and separately,
a system forecast vs. actual sales percent difference?

-No, I only have the resultant absolute percent errors.  The customer
gives us a forecast and the system generates a linear demand forecast
(for the same subset of parts) and both are compared to sales.  The
result of these two comparisons is an absolute percent error or APE
(say 50% and 79%).   All I have are the APE's and no raw data.  I'm
wondering when figuring out percent difference should I just subtract
one from the other or if that's wrong how should I handle it.

Also, what definition of percent difference do you want to use?  One
common definition is the difference between two numbers expressed as a
percentage of one of the numbers.  But there is another definition
used in science, "the absolute value of the difference over the mean
times 100," sometimes known as the "relative percent difference."

I'd be interested in both with a little explanation if you wouldn't
mind..I'll tip you a few bucks if it helps me once and for all
undetand percent increase/decrease and which number it should be based
on

THANKS!

---

Request for Question Clarification by efn-ga on 15 Sep 2004 19:13 PDT

Thanks for the clarification.  I understand the second answer, but not
the first answer.

Maybe it will help if I spell it out a little more.  Let's say we have
these quantities defined.

FC   the customer's forecast
FS   the system's forecast
AS   the actual sales
APEC = absolute percent error of the customer's forecast = ABS(FC - AS) / FC
APES = absolute percent error of the system's forecast = ABS(FS - AS) / FS

The percent difference should be a function of two numbers.  Let's
call it PD(x, y).  I asked whether you were looking for PD(FC, AS) and
PD(FS, AS), and you said no.  I asked what the inputs to PD should be,
and I don't think you answered explicitly.

It looks like you are interested in PD(APEC, APES).  Since APEC and
APES are numbers, a percent difference could be calculated, but I
don't think the result would be useful.  The question is really what
you are trying to accomplish by calculating the percent difference. 
Are you trying to compare the relative accuracy of the customer and
system forecasts, so a low percent difference would mean they were
about equally accurate, while a high percent difference would mean
that one was much more accurate than the other?

That's just a guess.  The short form of the question is "percent
difference between what and what?"

Also, if you would like an evaluation of the other person's
calculation method, it would help if you specified what it is.

--efn

---

Clarification of Question by massradius-ga on 16 Sep 2004 16:28 PDT

Maybe it will help if I spell it out a little more.  Let's say we have
these quantities defined.

FC   the customer's forecast
FS   the system's forecast
AS   the actual sales
APEC = absolute percent error of the customer's forecast = ABS(FC - AS) / FC
APES = absolute percent error of the system's forecast = ABS(FS - AS) / FS

The percent difference should be a function of two numbers.  Let's
call it PD(x, y).  I asked whether you were looking for PD(FC, AS) and
PD(FS, AS), and you said no.

I apologize.  I am looking for the PD(FC, AS) and PD(FS, AS) but I
don't have the raw data I only have the resultant APE.  The inputs
would be the PD(APEC, APES) (as that's all we have).  I realize this
isn't the best way to do things...out of curiousity how would you go
about it?


The question is really what you are trying to accomplish by
calculating the percent difference. Are you trying to compare the
relative accuracy of the customer and system forecasts, so a low
percent difference would mean they were
about equally accurate, while a high percent difference would mean
that one was much more accurate than the other?

You are correct.  I'm trying to compare the relative accuracy of the
customer and system forecasts

Also, if you would like an evaluation of the other person's
calculation method, it would help if you specified what it is.

Her method is to calculate percent difference by:

(APEC-APES)/APEC

Since all we're left with is perecents and no date I thought:

APEC-APES would be just as telling

---

Clarification of Question by massradius-ga on 20 Sep 2004 14:51 PDT

EFN,

I'm pleased with the answer...you have my permission to move it over. 
Just a few points of clarification:

1. Could you go over the scientific method in a little more detail
(i.e. the formula and short description of when this method makes more
sense)

2. If I had the raw data (i.e. SALES, FS, FC) would i be able to come
up with a meaningful percent difference

thanks a bunch!

Hi massradius,

Thanks for accepting my tentative answer, which appears below as a
comment.  Here I will address your request for clarification.

The formula for what I called the scientific method of calculating
percent difference in a programming or spreadsheet style of notation
would be

ABS(A - B) / ((A + B) / 2)

In the first example in my comment, the input values are 9900 and 50,
so applying this formula gives

ABS(9900 - 50) / ((9900 + 50) / 2)

= 9850 / 4975

= 1.9798994...

This method would be appropriate for measuring how close together two
approximate measurements of the same quantity are.  In the absence of
any information about what the value being measured really is, this
formula just assumes that it is right in the middle of the two
measurements.  For example, if you have two thermometers side by side
and one reads 99 degrees while the other reads 101 degrees, the
formula says the true temperature is right in the middle, 100 degrees,
and so the percent difference is 2%.  Similarly, if the thermometers
read 90 and 110, the percent difference would be 20%.

So this method doesn't really make sense in your situation, since the
two absolute error percentages you have are not really different
approximations of some common underlying absolute error reality.

In response to your second question, yes, if you had the raw data, you
could come up with a meaningful percent difference, at least in a
sense.  If you calculated error percentages using the actual sales
instead of the forecasts as the denominator, then both would have the
same denominator and it would be meaningful to subtract one from the
other.  In fact, you wouldn't even need all the raw data.  If you just
had absolute error percentages like you have now, except calculated
with the actual sales as the denominator, it would be meaningful to
subtract one from the other.

I have to qualify that, though.  That might be considered a percentage
point difference rather than a true percent difference.  It's a
problem like when something increases from 5% to 10% of some other
quantity.  Did it increase 10 - 5 = 5%?  Or did it increase 100%
because it doubled?  Some might use whichever interpretation supports
their case--the same change might be described as "a meager 5%" or "a
whopping 100%," depending on the speaker's agenda.  A careful writer
might describe the increase as "5 percentage points" to distinguish it
from the more ambiguous "5 percent."

Similarly, a percent difference between two numbers usually has one of
the numbers as the denominator, e.g., the percent difference between
90 and 100 is either 10% or 11.111...%, depending on which one you
choose as the denominator.  But if you subtract error percentages as I
suggested, the denominator is not either of the percentage numbers,
it's the actual sales.  So the difference is not exactly a percent
difference in the usual sense.  However, I think it would provide a
meaningful measure of the variance between the forecasts.

There are probably more sophisticated statistical methods of comparing
the forecasts, but they are beyond my expertise, and, I hope, beyond
the scope of this question.  I am fairly confident of my original
assertion that you can't get a meaningful percent difference with the
inputs you have.

--efn

Hi massradius,

I'm not quite confident enough of my opinion to post it as an answer,
so I'm posting it as a comment.  If you say it's worth the price you
have offered for the question, I'll be happy to post it as the answer.

I don't think a meaningful percent difference can be calculated from
the inputs you have.  The problem is that the inputs are the values of
fractions with different denominators, so they measure different
units, and the denominators are unknown, so you can't convert them to
a common denominator.

I'll try to illustrate with a few extreme examples.  In this
discussion, the typical percent difference method is the one used by
your colleague, (a - b)/a.  We don't have any particular reason to
choose APEC or APES as the denominator (at least, I don't), so I will
calculate it both ways.  The scientific percent difference method is
the one I mentioned in one of my Requests for Clarification, the
absolute value of the difference divided by the mean.

Let's say FC = 1, FS = 200, and AS = 100.  Then APEC = 9900% and APES
= 50%.  By the subtraction method, the percent difference is 9850%. 
By the typical percent difference method, the difference is either
about 99% or 19700%, depending on what you choose as the denominator. 
By the scientific percent difference method, the difference is about
198%.

Then let's say the next month FC = 5 and FS and AS are the same.  APEC
then is 1900%.  By the subtraction method, the percent difference has
decreased from 9900% to 1900%.  By the typical percent difference
method, the difference either changed from 99% to about 97% or from
19700% to 3700%.  By the scientific percent difference method, the
difference decreased from 198% to about 190%.

Now consider a month where FC = 100 and FS and AS are again the same. 
APEC then is 0.  By the subtraction method, the difference is 50%. By
the typical percent difference method, the difference is either 1 or
incalculable. By the scientific percent difference method, the percent
difference is 200%.

These examples show that all the methods are flawed.  The subtraction
method generates a huge change in the percent difference when FC makes
a relatively trivial change from 1 to 5.  With the typical percent
difference method, if the input you choose as the denominator is
small, it can generate a huge percent difference, even if the inputs
are not very far apart.  And the scientific percent difference method
generated almost the same percent difference in the first case, when
FC was 1% of AS, and the third case, where FC was exactly equal to AS.

If you want to compare the accuracy of the forecasts, you can look at
which of the absolute percent errors is bigger.  But I don't think you
can calculate the difference between them in any useful way.  I can't
prove this, though--there may be some ingenious way that just hasn't
occurred to me.

--efn