Google Answers Logo
View Question
 
Q: mystery shoper research progreamme ( Answered 5 out of 5 stars,   0 Comments )
Question  
Subject: mystery shoper research progreamme
Category: Business and Money > Consulting
Asked by: unico67-ga
List Price: $10.00
Posted: 06 May 2003 21:23 PDT
Expires: 05 Jun 2003 21:23 PDT
Question ID: 200451
I am looking for information on how to conduct a "mystery shopper"
research programme to test customer service, reception skills, product
knowledge etc of the retail staff in several retail shops in
Europe.Sector is luxury goods  - especially fashion.I am looking for
help on detailed methodological approach.
Answer  
Subject: Re: mystery shoper research progreamme
Answered By: tehuti-ga on 07 May 2003 06:40 PDT
Rated:5 out of 5 stars
 
Hello unico67,

I have summarised some of the salient points from the sources I found
and also added comments arising from my own experience as a mystery
shopper:

1. Matters relating to staff being assessed:

A mystery shopping exercise should aim to collect information that
will assist in improving staff training and retraining, the level of
service and customer satisfaction. It should not be used in order to
punish or dismiss staff.  The Guidance Council guidelines even go so
far as to say that mystery shopping “ is not usually designed to
measure product knowledge or to ‘test’ staff beyond testing
their customer service skills” and it stresses that mystery shopping
results should be seen as being quite different from the feedback of
genuine customers.

In order to ensure that the procedure is ethical. staff should be
warned that mystery shoppers will be used at times to monitor
performance. You will need to check on the legal situation in the
country in which the exercise will take place.  In the UK, the Data
Protection Act 1998 and Human Rights Act 1998 actually make it a legal
obligation to inform staff that they can be subjected to such
assessment. This can be done as part of the employment contract, in a
staff handbook, in newsletters etc.  It is not necessary to say
exactly when and how the asssessment will take place, but I know that
some companies do actually tell the staff the day on which a mystery
shopper will visit.

The document makes the point that staff should be told that it is not
acceptable to try to spot mystery shoppers, because they risk annoying
genuine customers.  It is useful to point out how the exercise can
benefit staff as well as management

If the scenario does not involve a purchase, and if you are assessing
staff who are paid on a commission basis, it would be fair to estimate
the time a member of staff will spend with a mystery shopper and
compensate them for the potential loss of commission.  If you are
assessing a competitor, where such compensation is not possible, you
should make sure that a non-purchase scenario does not keep a staff
member any longer than a normal customer enquiry. When assessing
competitors, you must not have any questions that would result in the
identity of a staff member being recorded.

2.  Matters relating to the mystery shoppers

You need to determine how you will recruit your shoppers.  What
profile are you looking for?  Are you going to place advertisements in
relevant publications?  How are you going to keep in contact and
collect data from your shoppers?  You mention that your survey is of
the luxury goods sector. You will need to find people who can appear
credible as typical customers of such establishments.  My own mystery
shopping is done on a casual basis, more for fun and because I like to
know I can help to influence things so as to get better service in
future.  Otherwise, I am in full time professional work.  One mystery
shopping company used to have the bad habit of assigning me a time at
which they would telephone to obtain my results, but they would
sometimes phone up to an hour earlier or later, which was highly
inconvenient.  I personally am happier with a system whereby I can
enter my data on a web site or send in completed questionnaires by
snail mail.

Your shoppers will need both general and project-specific training. 
How will you deliver this?  How will you check the training has been
understood?  The company I work for has a short quiz for each project
on its web site.  When I agree to a project, I have to read the
training notes and then do the quiz online.  I am only assigned the
project once I have submitted correct answers. General training
includes, for example, how to fill and submit questionnaires, what to
do if challenged by a staff member and accused of being a mystery
shopper, what expenses to claim and how to do this.

Reimbursment of expenses: this needs to be done expeditiously.  Also,
you have to decide what reimbursement is reasonable for your chosen
scenario.  Are you going to pay travel costs, or will you expect to
recruit shoppers locally?  Below, in part iii of the section on
scenarios, I describe a project in which I was forced to spend my own
money in order to maintain credibility.  As a result, I am not likely
to accept a further project in this series.

3.   Scenarios

Before starting, it is vital to define clearly the objectives of the
exercise, so that they can be expressed in simple and measurable
terms.

The scenarios used must be:

i. Relevant: “scenario must be designed to test the specific sales or
service behaviour that is the topic of study.  Think of the training
or instruction that staff have been given on how to deal with a
situation and come up with a scenario that, when delivered credibly,
should result in staff delivering those desired actions.”

ii Credible: it has to reflect the normal customer behaviour that
would occur. “While mystery customer researchers must be thoroughly
briefed, they must not appear too slick or over-rehearsed.  Overly
sophisticated scenarios can result in the staff guessing that the
mystery customer researcher is not a real customer, compromising the
value of the study”  The scenario needs to be one that the mystery
shoppers can understand.  The mystery shoppers need to be given a
detailed briefing so that they have background information to let them
develop their roles and so that they know how to deal with any
follow-up questions.  It is also important to make sure that mystery
shoppers used for a project fit the profile of customers using the
service that is being tested.
I personally have experience of a ongoing, continuous project of
branch visits and re-visits, in which the mystery shopper always has
to ask two complicated, obscure and totally unrelated questions, as
well as ask the counter clerk to perform one specific task. I imagine
that employees of this organisation have caught on to this pattern and
can identify mystery shoppers with ease!
Credibility also means ensuring that visits occur at different times
of day and on different days of the week, so as to get a good
reflection of staff performance in both busy and lax times. Obviously
this means making sure that the mystery shoppers have a space on their
questionnaire to indicate date and time of the visit.

iii  Practical for the mystery shoppers:  We have to remember our
answers to every point of the questionnaire until we have left the
premises and found a safe place to record our answers without the
possibility of being seen by any of the staff we have assessed.  Keep
the questionnaire simple and focused on the areas that are most
important to you. Another of my projects involved me having to check
on 10 (!!) different products, to ascertain whether each was being
sold at all, and whether it was displayed correctly. These products
are usually found on the shop counter. Imagine the problems of trying
to look at the things without staring fixedly at one spot for a long
period, while at the same time giving the impression of just having
come in for a quick purchase, and then having to remember all those
details for at least the next 10 minutes or so.  In order to buy
myself the time, I did have to make some purchases, however I had to
do this at my own cost.  This raises the importance of determining
what expenses are to be paid to the shoppers. If I had walked into the
store, approached the counter, stared fixedly at the relevant display
and walked out again, I surely would have blown my cover.  However,
the project designer had obviously not realised that I would need to
make a purchase to stay credible.
 
iv. Safe: “researchers are not asked to do anything illegal or that
puts them under any physical risk or threatens their personal safety
e.g. locality, disability, gender, ethnicity.  Care must be taken to
protect mystery customer researchers from any adverse implications of
carrying out an evaluation e.g. personal safety, effect on credit
references.”

v. Objective: most of the questionnaire should focus on factual
information. “The primary aim is to document precisely what happened
at the point of contact, rather than how the mystery customer
researcher feels.  Objectivity will also help ensure consistency
across all of the evaluations conducted.  However, some subjective
ratings, such as the perceived confidence of staff and the mystery
customer researchers overall satisfaction with the way their enquiry
was handled, may be useful when interpreting the results.”  In order
to maintain consistency, shoppers should be given minimal freedom
necessary to carry out their task.

3. Quality Control

Validation of data in questionnaire answers by means of computer and
logic checks is an important factor, and is a requirement, for
example, of the Market Research Quality Standards Association (MRQSA)
Service Standard, which led to the development of the current revision
of British Standard BS 7911 “Specification for Organizations
Conducting Market Research” Although this is for companies providing
market research services to other companies, it might be of relevance
to you in that it does describe best practice. “In particular, minimum
service levels are specified for quality assurance, executive
responsibilities, data collection and data processing to be applied in
market research projects.” The Standard can be purchased for £106 from
the British Standard Institution
http://www.bsi-global.com/Portfolio+of+Products+and+Services/Management+Systems/New/bs7911.xalter
A summary of the changes in the new revision is available at
www.bmra.org.uk/documents/285.doc
(Word document)

How will you check the assignment has actually be carried out in the
correct place? One way is to require the shoppers to send in some
concrete proof, eg a receipt, or a leaflet, business card or other
item that can be picked up in the store.  I am also frequently asked
to provide a brief description of the surroundings of the
establishment, eg names of the shops on either side.

5.  Costs

The Guidance Council document referenced below provides some estimates
of the direct costs of a mystery shopping project, given in UK £. 
This is in addition to the indirect costs involved in study design,
recruitment and training of shoppers, coordination of the project and
analysis of the results:

“If contracting for all, or some of, the elements of the research….
difficult to give any guidelines on this, as it will depend on the
methodology used and the extent of external involvement. As a guide,
in addition to negotiated development and set-up costs, the unit cost
per ‘mystery shop’ will be around £100… It is usual to pay mystery
shoppers an incentive payment… likely to be in the range of £10-£20
per shop plus reimbursement of expenses. …Payment for attendance at
any training or debriefing sessions should also be considered.”
NB In my experience, incentive payments are rarely as high as £20!!!

4. Other factors

Peter Gurney (see ref below) makes the point: “Mystery shopping is a
highly labor-intensive activity, and the vendors who supply this
service typically possess considerable logistical acumen. Their skills
lie in the hiring, scheduling, deployment and management of a
distributed workforce. Unfortunately, they're typically less
sophisticated when it comes to other important factors… such as
research design and analysis, development of enterprise-wide service
initiatives, integration of data across methodologies, and the linking
of performance data to incentives and training.”

Do you have all the above-mentioned skills available in your
organization?

And he makes another important point: “Operations, Marketing, Training
and Human Resources may all end up running overlapping service quality
programs that provide redundant data without being effectively
integrated with each other….  company executives have a tendency to
launch big service initiatives without thinking them through.
Companies may be surprised when their mystery shopping results don't
correlate well with customer satisfaction, increased revenue or lower
costs. But unless these variables are factored into the program design
from the beginning, it's unlikely that strong correlations will
result.”

and a warning: “Few companies anticipate the amount of administration
necessary to run a successful mystery shopping program. A common
mistake is to assign young, inexperienced personnel to handle the
program -- a sure prescription for chaos. It requires a strong
administrator to keep the company focused and on board, and to make
sure that recalcitrant field managers are not able to undermine the
program before it stabilizes and begins to realize its potential
value.”

There are advantages and disadvantages to managing a mystery shopping
programme for your own organization, using your own staff as shoppers,
compared with contracting it out to another company.
Advantages include: cheaper, more control, can be sure shoppers match
the required profile, staff can feel a greater ownership of the
project and be more willing to cooperate.
Disadvantages include: the full range of appropriate skills might not
be available in-house, need to assign staff to the project, shopper
anonymity could be compromised.
These form the two extremes of a number of possible approaches, which
are outlined in the Guidance Council document referenced below.

Mystery Shopping Providers Association (MSPA) has a Code of
Professional Standards and Ethics Agreement, both for service
providers and for shoppers:
http://www.mysteryshop.org/europe/ethics.php

Sources

The UK Market Research Society has published an 11-page document of
best practice guidelines on mystery shopping, which can be downloaded
in pdf, rtf or Word format at
http://www.mrs.org.uk/standards/mystery.htm

When Mystery Shopping Goes Bad by Peter Gurney. Reprinted from Quirk's
Market Research Review, February, 2002
http://www.kinesis-cem.com/Insights/mystery.html 

THE GUIDANCE COUNCIL
GUIDELINES FOR SETTING UP MYSTERY SHOPPING SCHEMES
http://www.guidancecouncil.com/download/mystery_shopping/mystery_shopping_guidelines_march_2001.PDF
Although this is aimed at assessing services providing information,
advice and guidance, many of the guidelines are also applicable to
mystery shopping in retail outlets.

ESOMAR Guidelines on Mystery Shopping
http://www.esomar.org/index.php (select Codes and Guidelines)
ESOMAR is the World Association of Opinion and Marketing Research
Professionals.


Further Reading

There are a number of articles and studies on the topic.  Perhaps your
library can help you to obtain some of these:

Burnside A., (1994), In-store spies snuff out poor service, Marketing,
April 28, 32-33.
Cobb R., (1995), Magical mystery lure, Marketing, October 26, 45.
Cramp B., (1994), Industrious espionage and Witnesses for the defence,
Marketing, August 18, 17-18.
Dorman K.G., (1994), Mystery shopping results can shape your future,
Bank Marketing 26 (8), 17-21.
Dwek R., (1996), Magic of mystery shopping, Marketing, October 17,
41-44.
Eisman R., (1993), Going undercover, Incentive, 167 (5), 70-71.
Finn A., (2001), Mystery shopper benchmarking of durable -goods chains
and stores, Journal of Service Research, 3(4), 310-320.
McNerney D.J., (1996), The link to customer satisfaction, HR Focus, 73
(9), 1-4.
Miles L., (1993), Rise of the mystery shopper, Marketing, July 29,
19-20.
Morrison L.J., Colman A.M. and Preston C.C., (1997), Mystery customer
research: processes affecting accuracy,  Journal of the Market
Research Society, 39 (2), 349-361.
Wilson A.M., (1998), The role of mystery shopping in the measurement
of service performance, Managing Service Quality, 8 (6), 414-420.

Search strategy: 1. "mystery shopping" methodology 2. "mystery
shopping" guidelines
unico67-ga rated this answer:5 out of 5 stars

Comments  
There are no comments at this time.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy