Google Answers Logo
View Question
Q: My website has been deleted from Google's data base. ( Answered,   0 Comments )
Subject: My website has been deleted from Google's data base.
Category: Computers > Internet
Asked by: jtfbus-ga
List Price: $50.00
Posted: 20 Jun 2003 12:25 PDT
Expires: 20 Jul 2003 12:25 PDT
Question ID: 219744
My website has been taken out or deleted from the Google's
data base (index file). I need to know the reason which prompted the
site deletion. Is there some thing I need to revise or else? The site
has been listed and was indexed for the past years.
Subject: Re: My website has been deleted from Google's data base.
Answered By: mcfly-ga on 21 Jun 2003 06:20 PDT
Hi JTFBus!

Initial investigations into yielded no insight into why
it should have been removed from the Google database, indeed the site
appears to be well designed with useful content and not too much
unnecessary code.  This led me to look for causes not dependent
specifically upon itself, but which were related to the
effects other sites were having upon it.  It was at this point the
solution to your problem became clear; the use of multiple duplicate
sites is a tactic which Google regards as a form of spamming, and is
likely to punish with exclusion from their listings.

Performing a search for 'jtfbus' enabled me to identify the following
sites which have substantially duplicated content from

Referring to Google's webmaster guidelines, the penultimate bullet
point in the quote below covers our area of discussion.

Google Information for Webmasters
Quality Guidelines - Specific recommendations:

    * Avoid hidden text or hidden links.
    * Don't employ cloaking or sneaky redirects.
    * Don't send automated queries to Google.
    * Don't load pages with irrelevant words.
    * Don't create multiple pages, subdomains, or domains with
substantially duplicate content.
    * Avoid "doorway" pages created just for search engines, or other
"cookie cutter" approaches such as affiliate programs with little or
no original content.

So it would seem that Google have detected a breach of their
guidelines, and have therefore removed from the search
database.  You may now be pondering over why it has happened now and
not before.  This is most likely to be related to the increase in use
of site duplication over the last couple of years; the following quote
makes the point concisely, if somewhat bluntly:

"the more that the search engine databases get compromised by
duplicate content, the more the engines will fight against it"

Search Engine Guide: Doorways, Mirrors, and Duplicate Sites

Thus far, my answer may have seemed a little negative, but hopefully I
can remedy this now by making explaining how you could resolve the

There are essentially two ways to start making more
'Google-friendly' again.  These involve either removing all the
duplicate sites, or making the duplicate sites invisible to search
engines.  If a substantial quantity of your business comes from the
duplicates, it may be desirable to leave them in place and use the
latter of the above options.

When Google, or any other search engine, records information from the
Internet, it does so with a piece of software called a 'spider'.  The
spider travels around the 'net through hyperlinks, just as a human
reader would do.  When the spider comes to a new page, it will check
the robots.txt of the site (to be discussed in a moment), and in most
cases will record the content of the page in the Google database.

It is possible to prevent a spider from indexing a site, effectively
making it invisible to the search engine, by using a specific command
in the robots.txt.  Currently the duplicate sites do not have a
robots.txt, but it is very simple to create one:  just make an
ordinary text file with notepad or similar, enter the following
commands, save it as robots.txt, and upload it to the duplicate site
root directory eg.

To stop any search spider from indexing a site, use the command:

User-agent: *
Disallow: /

Or to stop just the Google spider, use:

User-Agent: Googlebot
Disallow: /

Further information relating to the use of robots.txt is available

Creating and Using a robots.txt File

Having made these changes, it will probably take up to a month for
them to have an effect on the Google database.  However, once they
have, you will no longer be infringing the Google guidelines, and an
email to explaining the situation and specifying you
have disallowed the duplicate sites should result in
being re-listed in the results pages.

Below are links to a few sites which I have found useful when
researching the subject of search engine optimization.  They may be of
interest if you would like any further details about the function of
search engines.

Search Engine Guide

Search Engine Watch    
Webmaster World   
Search Engine World  

I hope you have found this answer both interesting and informative; if
you would like clarification of any area, please do not hesitate to
ask before rating my answer.

Kind regards,

mcfly-ga :)

**Search Strategy**

"jtf business systems"
google duplicate OR duplicated sites
google duplicate OR duplicated sites
google robots.txt
There are no comments at this time.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  

Google Home - Answers FAQ - Terms of Service - Privacy Policy