Dear colinduff71,
Interesting question! It caught my eye because my day job is very
mathematical and I have a strong interest in statistical modeling. Let
me address the two parts of your question in order.
I am a bit stumped as to what to do with answers of don't know/not
applicable or simply missed questions.
If a question has been skipped on a given copy of the survey by the
respondent, then you should assume in your analysis that the question
was not administered to that respondent. Since data from Likert-scaled
questions is not easily combined, you will be plotting the results for
each question separately. Thus, it doesn't matter whether you end up with
a different number of responses to each question over the whole survey.
If you want to be perfectly precise, you can note in your analysis that
your analysis of each question considers the aggregate Likert scores
obtained from those respondents who chose to answer the question. This
is generally implied in the survey methodology, but if you find that
there is an unusually high proportion of non-respondents for a given
question, it would make sense to note this in your report. It is quite
normal for the non-response rate to increase gradually over the length of
the survey. One way to deal with this problem is to randomly arrange the
questions, to the extent that this is feasible, on individual copies of
the survey. If you still find significant variation in the non-respondent
rate, your readers will be interested in a chart showing these rates. You
should still treat the per-question analyses separately, however.
Now let's take the case of "don't know" and "not applicable"
responses. There are two things to bear in mind here. First, if you
are the author of the study, then you are responsible for composing the
questionnaire instructions and setting the range of Likert values. You
don't have to include "don't know" or "not applicable" unless it is a
reasonable answer to the question that can be compared to other values
in the restricted range of responses. Second, the numbers assigned to
each response are largely arbitrary values that can't be compared in
the same way as inherently quantitative values. For example, it makes
no sense to say that a Likert value of 4 is twice as high as a Likert
value of 2 on a given question. This is why the preferred methodology
in analyzing Likert-scaled questions is to use the mode of the aggregate
scores rather than the mean or median.
To return to the matter of composing instructions, one way to handle the
"not applicable" response is to leave it out entirely. Then you can either
include at the beginning of your survey the explicit instruction that the
respondent should omit a response to any question that does not relate
to him or her; these question instances then join the non-respondent
pool. Alternatively, you can leave this point unsaid and simply assume
that respondents will skip questions that they cannot answer. The usual
way to maximize the applicability of the Likert scale is to include the
response "indifferent" or "suspend judgment" as the middle value. This
gives people who don't know or don't care about the question a way to
answer the question, yielding a lower non-respondent rate and a stronger
analysis.
I also don't know whether scoring on a 1 to 5 scale should be
counted for aggregation purposes as a 0 to 4 scale. it seems
to me if I count it as 1 to 5 then no answer can get any less
than 1, i.e. everything on a graph produced from ther results
actually starts at 1 not 0.
This question sounds more complicated than the one above but has a much
more straightforward answer. The key is to understand that the responses
in a Likert scale are not chosen for their numerical value but solely
as a convenient way of ordering them in the respondent's mind. What
counts is really the order and the uniqueness of each answer, not their
arithmetic properties relative to one another. In fact, you can use a
Likert scale that doesn't even have numbers, but the letters A through
E or some other set of ordered symbols.
In this light, the scale consisting of symbols 1 through 5 is equivalent
to that containing the symbols 0 through 4. One scale is simply a
one-to-one mapping of the other. There is no statistical significance
to which one you choose to represent the possible responses, although
1-2-3-4-5 is the more traditional scale and is therefore less likely
to raise questions in the respondent's mind. The trouble with zero is
that it is psychologically different from the positive integers, and
is therefore best avoided on the questionnaire if you wish to avoid
influencing your respondents.
But after you have collected the data, whether you employ the symbols
0 through 4, 1 through 5, or A through E to represent the underlying
responses makes no difference to the shape of the aggregate charts. If
some software package displays an axis that doesn't extend exactly from 1
to 5, say, you will still be able to adjust the axis endpoints or change
the type of plot -- try bar charts instead of line or point plotting --
to improve the aesthetics. This may improve the aesthetic presentation
of your results but not their statistical significance.
To recapitulate, skipped questions should be treated as questions
that were not administered to the respondent. You should then plot
aggregates separately for each question, and as an option, in your final
report you can include a separate chart of the non-respondent rate per
question. Responses such as "don't know" and "not applicable" can either
be avoided or incorporated into the Likert scale along with the other
questions. If you are not responsible for composing the questionnaire,
then you should consider how these non-committal responses are used
in the survey: if they form part of the Likert scale, then you include
them in the aggregate along with all other response values. If they are
separate from the Likert scale, then treat these question instances as
ones that were skipped. Finally, the 1 through 5 scale is preferable
to 0 through 4 for psychological reasons, but it has no effect on the
validity of your statistical analysis. If the value 0 does appear in
your plotting software, you can get rid of it by adjusting the axes or
by changing the plot type.
I have enjoyed addressing this question on your behalf. If you feel that
any part of my answer requires correction or elaboration, please let me
know through a Clarification Request so that I have the opportunity to
fully meet your needs before you assign a rating.
Regards,
leapinglizard |
