Google Answers Logo
View Question
 
Q: a model to rate objects based on many criteria ( No Answer,   4 Comments )
Question  
Subject: a model to rate objects based on many criteria
Category: Science > Math
Asked by: gethelp-ga
List Price: $50.00
Posted: 07 Jun 2005 13:32 PDT
Expires: 07 Jul 2005 13:32 PDT
Question ID: 530481
Hi, I need to describe what I need in proper terms that can be used as
a specification for a math professional to find or design the right
model for me.

Then I actually need the model itself.

The budget is very limited, but I am very interested in doing it the best way.

Here is what I need in plain english. Let us say I need to buy a
computer or an appartment or to hire an assistant etc. I need to enter
many objects (of the same type) available on the market, specify my
preferences, quantify and prioritize them them and calculate what is
the best choice for me.

For example criteria for a computer would be size of RAM, size of HDD,
speed of CPU, price etc. For an appartment will be distance from a
perfect location, price, number of bedrooms, condition, noise, etc.
For an assistant I'll need to enter quantified knowledge in many
areas, friendliness, compatibility with me, easiness in communication
with me, with others, english skills, wage/salary expectations etc.

Then I need to rate them all based on my criteria and on the
importance that I give to each of those criteria.

The final result - I need to rate them all and find the best choice.

Please let me know if you have anything in mind or can recommend
something or can provide a complete solution, or may be if you can
recommend someone.

Request for Question Clarification by leapinglizard-ga on 07 Jun 2005 14:29 PDT
There are sophisticated mathematical methods for optimizing numerical
constraint-satisfaction problems, but it seems to me that your
constraints are largely subjective. Although you might be able to
attach numbers to some of the product criteria, the way in which you
value those numbers is itself subjective.

Thus, I would say that the proper approach in your case is to develop
a weighted rating scale for each product type. You would then choose
the product with the highest arithmetic average over all ratings. This
the model that Car and Driver uses to decide the winners of its
automobile comparison tests, for example, and the method used in many
Slate.com stories to find the best popcorn, the best pen, and so on.

If I were to point you to some of these stories and to develop one or
two examples of my own -- I could work out weighted rating scales for
a computer purchase or an apartment rental, say -- would you consider
your question answered?

leapinglizard

Clarification of Question by gethelp-ga on 07 Jun 2005 15:39 PDT
You are right, it is not just subjective, it is nearly 100%
subjective. Even wirh a computer, for me personally the size of RAM
may have much more value than of the hard drive, but for a stororage
computer it could be the opposite. If I could give a proper weight to
each value of each criteria, I could do the rest by myself. My main
problem is how to convert a relational human perception into a weight
of a criteria. And again, it is absolutely subjective. I will myself
pick criterias, then I will determine (just arbitrary assign) a value
to each of the criteria, for example on a scale 1 to 10 this
parameter's value (for me personally) is 7. etc. Of I can give an
exact number if given a range (like for amount of RAM if I say that I
am looking for a computer with RAM between 256Mb and 1Gb (that would
be the range) and the one that is in question now has 512Mb of ram -
it is clear that it can be converted to a scale from 1 to 10 for the
value of this parametyr.

The real trick is how to incorporate here priorities/weight of the
parameters/criteria themselves based on relational human thinking.

Here is an example. Let us say we are talking about hiring an
assistance. I can say that having good English is more important to me
than having computer skills. Understanding what I am saying is more
important than computer skills too but less important that English
scills. Having computer skills is 10 times more important than being
attractive. Experience is twice as important than compensation
expectations. Etc. So first I need to have a simple, "human" way to
formalize the scales (parameters/criterias) themselves. Then when the
scales are sorted out I can give each candidate a mark in each scale.

And the last part, I guess the most straight forward is to calculate
rating for each candidate based on that candidate's position on each
scale also taking in consideration the difference in weight of each
scale.

I need a mathematical formula and an algorithm/methodology how to give
waight to the scales. I will come up with a different set of scales
each time (for computer one set, for an assistance another, for an
appartment yet another). The system should force the user to answer
some questions or to make some choices or do whatever it takes (but
the simpler for the user the better) to come up with weight of each
scale. The rest I guess is trivial.

Now I cannot answet your question because I do not have enough info to
base my answer on. May be you can answer how far away what you
suggested from what I need. Please feel free to ask if I am not very
clear in my comments.

Request for Question Clarification by leapinglizard-ga on 07 Jun 2005 16:44 PDT
I believe our thinking is fairly close. The way I would model
attribute preferences is to attach a numerical weight to each
attribute depending on how important it is. The relative magnitude of
the weights should reflect the relative importance I attach to the
corresponding attributes. Then, after rating a candidate for each
attribute on numerically identical rating scale, I take the weighted
arithmetic mean of the ratings to determine my overall favorite.

To take your example, suppose I'm evaluating secretarial candidates
based on English fluency, computer skills, comprehension ability,
attractiveness, salary expectation, and employment experience. If the
least important of these is attractiveness, I'll give it a weight of
1. Since computer skills are 10 times more important, they get a
weight of 10, and so forth. I end up with the following weights. Note
that comprehension ability is more important than computer skills, but
English fluency is more important yet. Experience is roughly halfway
between attractiveness and computer skills in importance, and salary
expectation is twice as important as experience. These relationships
are based on the comments you made above.

attractiveness          1.0
computer skills        10.0
English fluency        15.0
comprehension ability  12.5
employment experience   5.5
salary expectation     11.0

Now I work out some way to rate each attribute on a scale of 1 to 10.
You can also use percentages or a 1 to 5 scale, as you wish, since
what truly matters is the final ranking. To rate attractiveness, I
suppose that the ugliest possible candidate is a 1 and the most
beautiful in the world is a 10. For employment experience, I'll say
that a novice gets a 1 while a lifelong secretary, with 50 or more
years under her belt, gets a 10. I can similarly scale the salary
expectation to a 1-to-10 range, while English fluency is more of a
subjective impression, and so on.

Finally, to compute the weighted average for a given candidate, I
multiply each of her attribute ratings by the attribute weight, take
the sum of all these, and divide the total by 55, which is the sum of
all the attribute weights. This gives me the weighted arithmetic mean
of the attribute ratings. The candidate with the highest mean score
wins.

It's a fairly simple scheme, but I think it's reasonable. Has this
been helpful to you at all?

leapinglizard
Answer  
There is no answer at this time.

Comments  
Subject: Re: a model to rate objects based on many criteria
From: manuka-ga on 07 Jun 2005 23:50 PDT
 
It seems to me that what gethelp-ga is after is more a way of actually
determining on an interactive basis what weights to assign to various
elements. (The ranking part is reasonably straightforward.) This makes
the process much more reusable.

Here's one approach I've seen used (by a bank). The various elements
are divided into groups of a few related elements. (In the bank's
approach there was also filtering based on which elements the user
said they wanted; you may or may not want to include this aspect.) If
there were two elements requested in the same group a slider was put
up to choose between them (1-5). If there were three they put up a
slider from each of the three possible pairs; presumably they did some
sort of fit to get a compromise set of weights. Obviously you don't
want to put up six sliders where there are four elements in the group
- too much effort for the user. Indeed, I found the bank's one
confusing enough because some of the elements I was being asked to
compare seemed to mean the same thing. So probably three-way
comparisons are the most that is practical with this approach.

Note that you can operate on a hierarchical level, getting the user at
each level to indicate the relative weights of various groups.

Probably a better way of organising it if you have larger numbers of
elements is to use a panel of sliders (much like the Windows volume
control panel). You can have half a dozen or so elements being
directly compared this way. You could again use a hierarchical
approach if you have many elements. For instance, for a computer you
could have main areas of (say) Performance, Storage, Expandability and
Price. Performance could be divided into CPU speed, HDD speed, amount
of RAM, etc. Storage could be divided into HDD size, CD/DVD readers,
CD/DVD burners. Expandability could cover things like USB ports,
expansion slots, spare memory slots, CPU upgradability, etc.

If you provide sensible defaults for the relevant areas within each
group, the user only has to rate the main headings against each other.
But they can also go into the details for each group to fine-tune the
results, if they want to.
Subject: Re: a model to rate objects based on many criteria
From: gethelp-ga on 08 Jun 2005 08:40 PDT
 
Thank you, this is much closer to what I asked. Eventually I will need
to be able to calculate a final result based on those choices, but I
assume the math behind it will be simple. This is a great approach for
large sets. What if I have only let us say 4 parameters for a computer
- RAM, HDD, CPU Speed, Price.

I was thinking of a bubble algorithm where a user is asked to compare
each parameter with every other, asking them not only which one in
each pair is more important, but also how much more important. Then
when I have all those quantified relations between parameters (scales)
what do I do with them?

This is ideal because humans can compare 2 objects much easier than 3
or more of them at once plus it is not natural for most of average
people to make a relatively correct numeric chart as leapinglizard
suggested.

Also my approach (bubble comparing each pair of the set) looks like
compete, but the more parameters(scales) are involved the more tedious
the process will be for the user.

Do you see any compromise between my complete and easy but very long
bubble approach and may be something else that is as complete and easy
but not so time consuming as bubble?

Also if you agree than a bubble comparison is good enough, I do not
know how to convert my bubble results into a weight chart as
leapinglizard showed above.

One more thing that confuses me. Let us say we are talking about CPU.
A CPU may be Intel or AMD. I may have a preference of one over
another. Then I am not absolutely sure that for a binary answer
(yes/no) one should be at one end of the scale, another one will be at
another end of the scale. Even if the weight of the scale is not big
enough, won't my slight preference in CPU (If I do not have strong
opinion about which one to choose, but I have a mild wish) affect the
result too much? And after all, how do I compare speed then? Let us
say there are computers with Intel CPU with different speeds and
others with AMD CPUs with their speeds too, but I cannot compare the
speed of AMD with the speed of Intel, the comparison of just that
number would not be correct. I can split all contestant computers in 2
groups, one with Intel, another group with AMD. Then compare 2 groups,
find the winner in both, and then what to do with 2 winners? It comes
down again to comparing apples to oranges. Not to mention that I would
not want to complicate te process by splitting a set of objects to
compare into groups. I just need to come up with how to rate such
different things.
Subject: Re: a model to rate objects based on many criteria
From: chrislbs-ga on 08 Jun 2005 09:39 PDT
 
The folks in marketing have a nice way of approaching this problem. 
This would be a conjoint analysis problem.

As a first step, you need to outline the attributes of the different
products you are choosing from and prioritize them to take the most
important attributes.  The more attributes you have, the more
questions you will have to answer later.  Next,  look at the different
levels for each attribute.  Again, you may need to bundle some of
these together if you think they are functionally the same.  Having
more than 4 levels for an attribute makes for more questions.

Instead of looking at an attribute scale where you say that "attribute
a is 20% of my decision and attribute b is 30% of my decision," the
conjoint derives these weights from responses to a set of questions.

In a conjoint design, you will be asked to make tradeoffs [choose one
or the other] or to state a scaled preference [1=X and 5=Y] between
Bundle X and Bundle Y.  Bundle X might be attribute A at level 1 and
attribute B at level 3.  Bundle Y might be attribute A at level 3 and
attribute D at level 2.  Through a careful experimental design, you
can ask a limited number of these questions to get at your preference
for each.  You can also determine the value of the difference from
going from one level to another for each of the attributes.

Finally, when you are done, you can evaluate your utility for each of
the many items that exist and then choose the one with the highest
utility.  This is a nice thing for product designers because they see
what is more important without having to have an "existing" product
out there.

A basic approach to this can be done with analysis from a spreadsheet,
but the experimental design part is probably best accomplished with a
good statistical program by an expert.

http://marketing.byu.edu/htmlpages/tutorials/conjoint.htm
http://www.quickmba.com/marketing/research/conjoint/
Subject: Re: a model to rate objects based on many criteria
From: turt-ga on 30 Jun 2005 09:29 PDT
 
Conjoint analysis is one way of approaching this kind of problems. I
can give you some more details as I wrote my master thesis about it.
Others are:
- multiple criteria decision analysis, i.e.:
http://www.sm.umist.ac.uk/wp/Papers/wp0205.htm
- Analytic Hierarchy Process i.e:
http://mat.gsia.cmu.edu/mstc/multiple/node4.html

Hope this will help you

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy