Google Answers Logo
View Question
 
Q: Google website indexing ( Answered,   0 Comments )
Question  
Subject: Google website indexing
Category: Computers > Internet
Asked by: wayne36-ga
List Price: $25.00
Posted: 14 Nov 2002 13:01 PST
Expires: 14 Dec 2002 13:01 PST
Question ID: 107861
Why was our company's website www.floridadriftwood.com dropped from
Google's index?  Even if we search the company name, it does not
appear.  We have been well listed with google for 3 years now and have
enjoyed high listing in our category for much of tat time.  We noticed
today that they completely dropped us from their index.  Was this our
fault?  How can we avoid this in the future?
Answer  
Subject: Re: Google website indexing
Answered By: gan-ga on 14 Nov 2002 22:16 PST
 
Hello Wayne,

I have spent some time analysing your site, and I believe your
non - listing situation on Google to be eminently reversible.
I realise you may well already have some expertise in the field
of site promotion and design; please skip over any sections you feel
you already understand.

If you would like any further information on any points, please do
not hesitate to ask me for answer clarification before rating my
research which follows.

Before I begin, I'll have to say that although I am a Google
researcher, that engagement does not provide me with any internal
information concerning their website ranking and selection technology.
Anything I mention here will be information gleaned from other web 
sources, texts and my personal experience in site promotion.

In order to avoid the problem in future, consider my suggestions
which are set out below under the following general headings:

1. Is there *really* a problem?
2. Previous promotion history.
3. External links.
4. Non - Google listings.
5. Server malfunction.
6. Robot - indexable content:
    6.1 Title content.
    6.2 <body> content.
    6.3 image and button content.
    6.4 <meta> content.
    6.5 robots.txt file.
7. Visual impact

1. Is there *really* a problem?
"""""""""""""""""""""""""""""""
From my own personal experience, I once lost a site completely in a 
similar manner to which you describe - it disappeared clean out of the
Google index. I spent a month laboriously checking every line of HTML,
every incoming link.. I thought *that* site had been banned from the 
index for some inexplicable reason. The site reappeared in the next 
update though, right as rain, with Page Rank up a point, even though
I had essentially left the site unchanged. Many factors are completely
outside of our sphere of influence as site designers, such as hosting
failure just at the wrong time, when the robot visits, etcetera.

It's thus sometimes worth waiting for six or seven weeks without
substantially altering your code, if you've checked the basics and
cannot identify any major problems, just to see what happens. It is
entirely possible that Google may relist your site on the next
monthly database update.

See:
Google information for Webmasters:
://www.google.com/webmasters/2.html#B1

Although I *did* find a number of changes you could try, in order to
improve your site ranking, I found obvious and solid reasons for a 
complete site removal to be substantially absent - hence my suspicion
that your non - inclusion at this time may well be not permanent, and
not due to some unspecified heinous HTML crime you or the designer
have comitted.

2. Previous promotion history.
""""""""""""""""""""""""""""""
Something I cannot tell just from looking at your site, or searching
the web, is the history of your domain in terms of what has occurred
previously with respect to site promotion efforts. It is possible to
cause a page to be dropped from the index by repeatedly submitting
it's
URL for inclusion. In the worst case scenario, this activity can lead
not only to that one page being dropped, but also any other pages from
that domain.

If you feel this may have occurred - for instance, if you have
previously engaged a third party to perform 'site promotion' or
'URL submission' services and discover that they have employed such
methods, it is sometimes possible to rectify the situation by sending
*just one* polite email to Google, explaining what you feel may have
happened, what you have done to rectify the situation (i.e. disengage
the service which was repeatedly submitting), and asking if your site
may be considered for re-inclusion.

You should contact Google in this respect according to:

://www.google.com/webmasters/2.html#B1

(See the link right at the bottom of the page).

I believe this to be an avenue of last resort, however.

3. External links.
""""""""""""""""""

Many search engines factor into their rating methodology, the number
and quality of links pointing at your page. This is widely held to be
very true for Google. If too few other sites link to yours, it is
taken
to represent a low popularity for your page.

On the other hand, there is a widely held suspicion that *too many*
links, especially ones from a large number of identical pages on
different domains, can have a damaging effect, in that this can signal
the use of artificial methods to attempt to 'hoodwink' the search
engines
into believing that the site is extremely popular. 

This can occur through the use of 'link farms', or services which act
to
automatically create a large number of inter-domain links by the
addition
of your URL to a stock 'link page', which all webmasters involved in
the
scheme keep on their server.

I have searched your domain for evidence of this activity, but have
not
found any - but it might be worth checking that you are not harbouring
any of these generic 'link pages' unwittingly - perhaps as a result of
a previously departed webdesigner or service. Unfortunately I can't
check definitively for you as this would require privelidged access
to your server.

I have performed a check to gain an idea of how many links you
currently
have pointing to your root domain; note that the service used could
itself
be considered a 'link farm', however the tool that the below URL links
to
is quite useful on it's own for a quick check:

http://www.linkstoyou.com/checklinks.htm

At the time I checked, I found that the results were as follows:

Hotbot search engine: found 28  links pointing at
www.floridadriftwood.com
Fast Search:          found 127 links pointing at
www.floridadriftwood.com
MSN Search:           found 35  links pointing at
www.floridadriftwood.com

This is actually quite low. When I visited a number of these links, I
found, however, that they were almost all good quality links from
relevant
sources. I believe you could considerably improve the perceived
quality
of your main page, viewed from a search engine robot's perspective, by
working on increasing the number of these links.

A technique I have found very useful is to search out sites which
complement yours, without being in direct competion, and email the
webmaster of each with a polite request to consider swapping links.
Most webmasters recognise the value of quality links and are eager to
work with you, especially if your site is of good quality, which yours
clearly is.

4. Non - Google listings.
"""""""""""""""""""""""""
Although your site is indeed not currently to be found in Google's
index,
it *is* well listed elsewhere. I visited Profusion's Meta Search,
which
simultaneously queries a number of different search engines, and
performed
a search on your main page title - "FloridaDriftwood Inc."

Profusion.com > Home > Web Search Engines:
http://www.profusion.com/default.asp?agt=0!&cat=1!&cobid=ess&sid={61A78DA7-27A2-4C89-9ECB-294210CF03C9}&queryterm=

I found that You have good listings on the following search engines:

All The Web
http://www.alltheweb.com/

Lycos
http://www.lycos.com/

Teoma
http://www.teoma.com/

MSN
http://search.msn.com/

WiseNut
http://www.wisenut.com/

What I did find striking, though, was the fact that not one listing
referred
to a standard .html page - all links listed were active server pages,
with
the file extension .asp

What is an ASP file?
http://www.aspronline.com/learn/lessons/1/1.asp

Having scanned random Google listings for examples of listed .asp
pages,
however, I conclude that this *may* not a problem, as many are
currently
listed. I am unaware wether Google considers .asp files to be 'dynamic
pages' however, and it is stated that only a limited number of
'dynamic
pages' are included in the Google index:

Reasons your site may not be included:
Your pages are dynamically generated:
://www.google.com/webmasters/2.html#usedtobe

Possibly you might consider running a static file 'index.html' as your
main
page - I was unable to locate such a file. I noted that you have a
redirect
occurring from index.asp to store.asp; This former may be being seen
by
robots as a 'doorway page', set up as a funnel for the primary purpose
of
routing traffic to a secondary page, thus causing your root domain to
remain
unlisted?

5. Server malfunction.
""""""""""""""""""""""
On several occasions, most notably around 03:20 GMT, I found the site
to be
slow to load or even totally inaccessible, with a 'Cannot find server
or
DNS Error'. If there has been appreciable server downtime, or
inaccessibility for another reason, the Google search robot may well
have
encountered difficulties when trying to crawl your pages on the last
update.
The remedy for this would depend upon wether you are running your own
server,
or wether you are using a third party to supply webhosting / routing
services,
to whom you might direct a query. You may wish to use a monitoring
service,
such as AlertSite:

http://www.alertsite.com/

to monitor your server's performance, or even just examine your
statistics
for unexplained periods of 'no traffic', in order to determine if
there is a
persistent problem in this area.

6. Robot - indexable content:
"""""""""""""""""""""""""""""

In this area, I have found that there is considerable opportunity for
making
changes which may positively impact your ranking. 

    6.1 Title content.
    """"""""""""""""""
    Currently, your main page title "FloridaDriftwood Inc." is quite
short,
    and does not contain any popular keywords. Although it will
certainly
    lead customers who already know of the existence of your company
back
    to your page, I doubt if it will generate any new traffic.
    
    Google certainly uses the title tag of the HTML page in it's
evaluation
    of the purpose of a site, so I'd suggest using one of the services
    available on the web to determine a few good search phrases
relevant
    to your business, and use them in your title tag. A very good
resource
    with plenty of information on this point is Robert Woodhead's
site,
    'SelfPromotion.com':

    http://selfpromotion.com

    6.2 <body> content.
    """""""""""""""""""
    In my own experience, I've found that Google indexes displayable
text
    placed between the <body> and </body> tags extremely well. Your
main
    page currently contains very little in the way of truly indexable
    body text - by far the greater proportion of the source code is
taken
    up by style sheet commands, HTML formatting etcetera. The
indexable
    body text that is present, is almost halfway down the page of
source
    code - lowering it's relative importance.

    I'd consider including in your page, a good descriptive couple of
    paragraphs. Whilst I wouldn't go 'over the top' trying to apply
    'search engine optimisation' techniques, which have a reputation
for
    backfiring, and stilting the readability of your text, I would
still
    attempt to ensure that the text you include, includes relevant
    keyphrases which are popularly entered into search engines by
    potential visitors.

    The body text should be kept, in the source code, as high as
possible -
    immediately after the <body> tag itself is ideal. You can position
the
    actual displayed text anywhere you like on your webpage:

    Enclose it within div tags, and use style commands to position
that div
    below your main, attractive design:
 
    <div id="info" style="container: positioned; position: absolute;
top: 600; left: 20">
    here's some text 
    </div> 
 
    This will ensure that the search engines, including Google, will
read your
    prose, and index the keyphrases within it, easily, whilst you are
able to
    retain substantially the same layout. (just adjust those top: and
left:
    parameters to get the effect you need.) 

    6.3 image and button content.
    """""""""""""""""""""""""""""
    You have a great opportunity within your image tags. Each one
should
    contain an 'alt' attribute, containing a keyword or phrase. Also,
if
    posssible, the image filename should tie in with this:

    <img src="florida flag fish.gif" alt="florida flag fish">

    This allows people who are searching by image on Google, to find
your
    page via that route - a useful extra source of traffic, also
helping
    your ranking relative to your competitors' pages. 

    6.4 <meta> content.
    """""""""""""""""""
    Your Meta keywords and description tags would possibly be more
effective
    placed up above the many lines of style sheet commands on your
main page.
    It is rumored that these tags are nowhere near as important as
they once
    were - <body> text is the important item. For more information and
to join
    the discussion if you wish, visit: 

    Webmaster world forums:
    http://www.webmasterworld.com/ 
    
    They keep a close eye on the search engines there, and are a
veritable
    goldmine of good, current information.
    
    6.5 robots.txt file.
    """"""""""""""""""""
    You have a robots.txt file on your server, a standard file which
is used
    to instruct visiting search engine 'spiders' as to which areas of
your
    site they are allowed to access. Your robots.txt file contains the
    following text:

    User-agent: *
    Disallow:

    This should allow all robots (*) access all areas (no entry for
Disallow).

    See:

    A Standard for Robot Exclusion
    http://www.robotstxt.org/wc/norobots.html

    Robots.txt Tutorial
    http://www.searchengineworld.com/robots/robots_tutorial.htm

7. Visual impact.
"""""""""""""""""
The actual visual impression given by the site is unimportant to
Google,
since no primary indexing is performed by a human. However, I feel
that
your design is very clear - I immediately knew the purpose of the site
as soon as I visited, and the navigational structure was also clear
and
self-explanatory. Obviously this is merely my own subjective
viewpoint,
but I don't feel improvement is needed here.

The one main situation in which I would reccommend someone to improve
their site visuals would be to assist in gaining a listing on the open
directory project, DMOZ:

http://dmoz.org/

Inclusion in the directory above does depend upon evaluation by a
human editor.

A link from the above can prove invaluable in alerting Google to the
existence of your site - however I have checked, and you are
already listed:

See DMOZ category Top:Shopping:Pets:Fish:Supplies:Plants


Search strategy:
Existing knowledge


Additional links:

Site Valet: Tidy Online: 
http://valet.htmlhelp.com/tidy/

Does your code test as valid HTML, at the W3C testing site: 
http://validator.w3.org/

Google information for webmasters: dos & donts 
://www.google.com/webmasters/dos.html 


I hope this information has helped. If you would like me to expand on
any point, or feel that there is something I have not covered, please
do not hesitate to ask for clarification before rating my answer.
 
Best regards, 
 
gan.
Comments  
There are no comments at this time.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy