QUESTION:
If search engines find multiple urls with the same content, do they
consider this spam?


BACKGROUND
We've hired a firm to optimize our web site. They recommend we change
the url
http://www.ssec.honeywell.com/microwave  to something like
http://www.ssec.honeywell.com/advanced-microwave-components-manufacturer
Rather than simply renaming the file (and creating a 404 Not Found
error), I proposed we create a symbolic link (symlink) so both urls
always return the same content.

They replied that Search Engines might view this as spamming.  I've
read Google's "Dos and Don't for Webmasters" 
://www.google.com/webmasters/dos.html  but this does not appear to
prohibit symlinks.  If Google does consider this spam, would listing
the old url in our robots.txt file
http://www.ssec.honeywell.com/robots.txt buy us forgiveness?  I am
really just
looking for the "right" way to retire a url without loosing traffic or
creating 404 errors.


FOLLOW-UP QUESTION
They recommended we use a redirect instead, and claim that a redirect
in less than 30 seconds would also be considered spam.
    <meta http-equiv="refresh"
content="30;URL=http://www.ssec.honeywell.com/advanced-microwave-components-manufacturer/">
I think 30 seconds is too long.  Are they right?

The exact methods used to refine search results are, for obvious
reasons, a trade secret. However, I can understand why two pages on
the same server with identical content would be considered an attempt
to artificially increase ranking. Ideally, the search engine should
only substantially decrease the ranking of all but one of these pages,
which would not influence the remaining page's ranking. A redirection
*page* is similar, as in the end, you have more than one web page with
the same ultimate content.

"I am really just looking for the 'right' way to retire a url without
loosing traffic or creating 404 errors."

Indeed you are. And the right way to do this is via an HTTP Redirect.
Instead of your web server returning a page, it returns a status code
that says "301 Moved Permanently". (RFC 2616, section 10.3.2:
http://www.ietf.org/rfc/rfc2616.txt )

The method for making this happen can vary between web servers, but
many allow you to add the following line to the ".htaccess" file in
your public web directory (you can create this file if it does not
exist):

Redirect permanent /microwave
/advanced-microwave-components-manufacturer

All browsers will instantly follow the link through, so you won't lose
any traffic.

This sort of thing happens all the time, and there is no reason for a
search engine to consider this spam -- it will follow through the
redirect as well (the prior page simply no longer exists), and only
index the new page.

---

Clarification of Answer by alexander-ga on 31 Oct 2002 18:24 PST

It appears that the "Redirect permanent" line got wrapped by Google.
It should be all one line, no matter the length, and simply include
the command "Redirect permanent" followed by a space, the old file,
another space, and the new file.

I can't say that I've EVER seen a redirect with a timer that long.

Typically, the redirect timer is set around 5 seconds or so, with a
simple message about "This page has been moved. If you are not
redirected to the correct link within 10 seconds, please <link>click
here</link>."

I don't see how EITHER solution would be considered spam, honestly.
Both pages are listed on the same domain, and would be categorized by
Google in a group; if both pages contain identical or virtually
identical content, Google would automatically crop one of the results
(whichever is older, i would guess) and only display the newer of the
two pages.

Spam, in my book, is unsolicited email. There are other similarly
aggravating offenses (pop-up, pop-over, pop-under, or flash ads, for
example) but having two URLs on the same server that display the same
content is certainly not one of them, IMO.

a redirect would be fine, so long as the redirect page doesnt have any
hidden text or metatags in it. it would also help to add the file to
your robots file. it doesnt matter how long the redirect. the symlink
would be considered spam by at least a few search engines, but i have
no idea about google (the most important one to get into besides
yahoo)