Is there any difference in the statistical quality of responses
between a four-point scale or a five-point scale?

---

Request for Question Clarification by mathtalk-ga on 19 Feb 2003 07:20 PST

Hi, mosesralph-ga:

I get the impression that you are designing a survey.  Survey results
can be processed using a variety of statistical techniques, depending
on the purpose for which the survey is taken.  While adjustments to
many of these techniques are required for 4-point vs. 5-point scales
(ordinal data), that would not be the most important aspect of the
design.

One purpose of surveys is often to track changing attitudes over time.
 In this case consistency of the rating scale would be important, to
avoid uncertainty over the participants interpretations of two
different scales.  Apart from this I would think the circumstances in
which a five-point scale "outperforms" a four-point scale are somewhat
specialized.  For example, we are accustomed to a five-point scale of
sorts (A,B,C,D,F) in educational contexts, so in related contexts a
similar scale might be helpful in promoting common understanding of
the scale.  In general a more detailed rating scale requires more
detailed instruction to survey participants in order to make the
increased choices meaningful.

Your question is quite brief.  If you would care to elaborate on the
issues you are working on, I'd be happy to try to provide an answer. 
What statistical tests do you plan on using?  Is there a "hypothesis"
that is being tested, i.e. is this survey part of an experiment? 
Otherwise what purpose is the survey to be designed for?

regards, mathtalk-ga

---

Clarification of Question by mosesralph-ga on 19 Feb 2003 08:54 PST

Excellent observations.
I have two relatively simple objectives at this point.
1. I need to convert some data from a 5-point scale to 4-point for
comparison. It occured to me that there could be some bias in that
process. Obviously all 5's covert quite cleanly to all 4's, but all
3's convert to 2.4.  From the other direction, there isn't a 2.4
choice on the 4 scale. So is a simple conversion formula (X/5.00*4.00)
a valid recipe?

2. I plan to adopt a 4pt scale to make it easier create surveys. We
attach a descriptive word or phrase next to the rating, something
beyond Excellent, Good, Fair, Poor.  To that end I was curious if
there is an arguement that challenges the simpler scale-that is beyond
your illustration of our educational template.

---

Request for Question Clarification by mathtalk-ga on 19 Feb 2003 14:04 PST

Let's drill down a bit on the situation where you need to compare two
sets of survey results, one involving a five-point scale and one
involving a four-point scale.

If the hypothesis to test were simply that the two populations
surveyed were not the same (that it is unlikely that two such samples
would have been drawn randomly from the same population), then I would
proceed to try and show that even by conservative methods the "null
hypothesis" is unlikely.

By this I mean that instead of trying to "map" the five-point scale
onto the four-point scale linearly (as your "(X/5.00*4.00)" formula
would try to do), I would instead lump categories together in both
surveys under favorable/unfavorable headings.  This would make for an
especially simple T-test for statistically comparing the samples.  If
the T-test shows small p-values for "lumped" observations, it
indicates conservatively that the sameness of sample populations is to
be rejected.

But perhaps you can explain more fully what the goal of comparisons
are, since I'm speculating here about what you are trying to
accomplish.

regards, mathtalk-ga

Greetings:

Scales and their use in survey research is a very complex topic. One
cannot freely assign the responses given to one scale to a different
scale without introducing bias.
The range of numbers, the use of negative numbers, words assigned to
numbers all affect responses in statistically significant ways.

You may wish to review some of the literature on the topic. For
instance:
Churchill Jr., Gilbert A. and J. P. Peter (1984). "Research Design
Effects on the Reliability of Rating Scales: A Meta-analysis." Journal
of Marketing Research, 21 (4), 360-375.

Friedman, Hershey H. and Taiwo Amoo (1999a). "Rating the Rating
Scales." Journal of Marketing Management, 9 (Winter), 114-123.

Friedman, Hershey H. and Taiwo Amoo (1999b). "Multiple Biases in
Rating Scale Construction." Journal of International Marketing and
Marketing Research, 24 (3), 115-126.

Goode, Erica (2001). "Researcher Challenges a Host of Psychological
Studies." The New York Times, January 2, F1-F7.

Haddock, Geoffrey and Rachael Carrick (1999). "How to Make a
Politician More Likeable and Effective: Framing Political Judgments
Through the Numeric Values of a Rating Scale." Social Cognition,
17(3), 298-311.

Schwarz, Norbert, Barbel Knauper, Hans J. Hipler, Elisabeth
Noelle-Neumann, and Leslie Clark (1991). "Numeric Values May Change
the Meaning of Scale Labels," Public Opinion Quarterly, 55(4),
570-582.

Schwarz, Norbert and Hans J. Hipler (1995). "The Numeric Values of
Rating Scales: A Comparison of their Impact in Mail Surveys and
Telephone Interviews," International Journal of Public Opinion
Research, 7, 72-74.

There are discussions of Likert scaling at
http://trochim.human.cornell.edu/kb/scallik.htm
and at
http://www.cultsock.ndirect.co.uk/MUHome/cshtml/index.html
and
http://www.icbl.hw.ac.uk/ltdi/cookbook/info_likert_scale/

A discussion of Thurstone scaling is at
http://trochim.human.cornell.edu/kb/scalthur.htm

And of Guttman scaling at
http://trochim.human.cornell.edu/kb/scalthur.htm

Also see:
Overcoming Scale Usage Heterogeneity:
A Bayesian Hierarchical Approach
Peter E. Rossi, Zvi Gilula,
and
Greg M. Allenby1
http://gsbper.uchicago.edu/respapers/rescaling/overcoming%20scale%20usage.pdf


Search strategy: "marketing research" questionnaire design scale;
instrument design scale; likert scale