Hello I am looking for a simple example which I can follow showing the Weighted
Least linear square method.  I am hoping to find an example I can
follow as I do not have a strong math background and I want to use
this for a 2 dimensional graph with about 5 results on it.

Thanks,

Justin.

Hi, Justin:

Is it fair to assume you want to apply the weighted least-squares
method to fit a straight line to some points on a graph?

regards, mathtalk-ga

Yes it is that is exactly what I want :-)

Thanks,

Justin.

Suppose the weights are w1, w2, ... the independent values are x1, x2,
... and the dependent values are y1, y2, .... Form the following sums:
I = the sum of the weights, II = the sum of the weights times the x's
(w1*x1+w2*x2+...), III = the sum of the weights times the squares of
the x's (w1*x1^2+w2*x2^2+...), IV = the sum of the weights times the
y's (w1*y1+w2*y2+...), V = the sum of the weights times the x's times
the y's (w1*x1*y1+w2*x2*y2+...). If m = the slope of the line and b =
its intercept, then they are the solutions to the following two
equations:

I*b + II*m = IV and
II*b + III*m = V.

The solution to these two equations is the following:

Let D = I*III-II^2. Then b = [III*IV - II*V ]/D and m = [I*V - II*IV]/D.

I'm surprized none of the paid answerers have responded to this
question. Hope this helps.

As berkeleychocolate-ga points out, the use of weights in a (linear)
weighted least squares fit amounts to introducing the weights as
factors on each corresponding term of a summation in the usual
formulas for an "ordinary" least squares fit.

Let's do an example to make sure the concepts are clear.  These
calculations are easily organized in a spreadsheet.  For example we
might put the weights w_i, the independent observed variables x_i, and
the dependent observed variables y_i in parallel rows:

  1.0    2.0    1.0    2.0    4.0   (weights)

  1.0    2.0    3.0    4.0    5.0   (x values)

  2.35   2.29   2.24   2.27   2.25  (y values)

This is a made-up example, motivated by a week in which a stock price falls.

The sums defined by berkeleychocolate-ga are now:

I          10
II         36
III       150
IV         22.71
V          81.39

which I used a spreadsheet to compute after adding rows for the
products of weights times x values, x-squared values, y values, and
x*y values (since "I" is just the sum of weights, it didn't need an
auxiliary row).

The denominator D = I*III-II^2 = 204, and:

y-intercept:  b = 2.34

      slope:  m = -0.02  (slight downward trend as expected)

Perhaps most illustrative is to compute the "model fit" values of y predicted:

  2.32   2.30   2.28   2.26   2.25  (y predicted)

and to compare these with our starting observations:

  2.35   2.29   2.24   2.27   2.25  (y observed)

Although the input data bounces up and down a bit after the first
"day", the larger weight assigned to the final observation (w_5 = 4)
resulted in a close fit to that point.


regards, mathtalk-ga

The coefficients of the line as I gave them above are rounded.  More precise is:

m = -0.01794

b =  2.33559

Adding another digit to the predicted y values may also clarify the results:

  2.318  2.300  2.282  2.264  2.246 (y predicted)
  2.35   2.29   2.24   2.27   2.25  (y observed)
 ------ ------ ------ ------ ------ 
 -0.032  0.010  0.042 -0.006 -0.004 (y errors)

So the observations with the larger weights in this case wound up with
the smaller errors.  Because the weights were all (positive) integers
in this problem, another way to formulate the example would be to
repeat the observations as many times as their weight and use the
"ordinary" least squares method.  Of course repeating terms in the
summations would lead in this fashion to the same calculations as
above.

For background note that justin_champion-ga has recently posted a
Question (Answered by elmarto-ga) about the ordinary least squares
method:

[Q: Least Linear Square Method simple example]
http://answers.google.com/answers/threadview?id=760922



regards, mathtalk-ga

trhank you very much for the replies I will try these out tomorrow
morning.  I again aplogise for not replying quickly as I had sort of
given up on getting a reply and have not checked often enough :-)

Thanks,

Justin.

Thank you every much the answer you gave was perfect and very clear. 
I have followed it along now and put it into Excel and it works
perfectly. :-)

Thank you very much for your time and effort, it is really appreciated.


Justin Champion