Google Answers Logo
View Question
Q: Url & Google Spiders: <a href="terms.html"> ( Answered,   2 Comments )
Subject: Url & Google Spiders: <a href="terms.html">
Category: Computers > Algorithms
Asked by: websitepromotions-ga
List Price: $30.00
Posted: 27 Nov 2003 09:14 PST
Expires: 27 Dec 2003 09:14 PST
Question ID: 281174

Is it important to have the full URL i.e.
Within the html code of the page so the search engines are able to
identify the link and crawl to the next page successfully?

I sometimes use a script driven navigation menu (mapped - by Microsoft
FrontPage) which when coded is like this <a href="terms.html"> and is
also displayed as the same when viewing the source code from the
online web page but returns results like the urls listed below when I
use a sim spider which is of course incorrect. Is this an error with
the sim spider or is it because <a href="terms.html"> is not
negotiable by the search engine spiders including Google.

(The / slash is missing after 

These should be:  

Hope someone can clarify this.

Subject: Re: Url & Google Spiders: <a href="terms.html">
Answered By: serenata-ga on 02 Dec 2003 18:22 PST
Hi Bruce ~

To answer your question about links and a sim spider --

relative links (which are links that do not contain the full URL but
just the individual page in relation to where they are placed) are
spidered just as well as absolute links (which are links containing
the full URL of a page).

I have a feeling you may have used a sim spider which contains errors.
I found one on the WebDevStore website, here:

whatever happens, it does produce an error when using "" in the
URL, although it doesn't seem to have problems with the "regular" top
level domains (ie., .com, .net, etc.) Maybe it's the two periods
because of ""

Previously, problems have been noted with certain sim spiders when
using relative links. For some reason, certain sim spiders do not
always realistically simulate how Google - or any other search engine
- would read the links, because Google and other search engines can
find and index relative links with no problems.

If you think about it - server side includes (SSI) and cascading
stylesheets (CSS) are fed via relative links; and if search engines or
browsers couldn't read relative links, they would not be able to
deliver your content as you want it seen.

I located a discussion from Webmaster World on just that subject -
errors in sim spider simulators - in a thread entitled "Sim Spider not
finding links - not finding internal links":

If you use the Sim Spider available on the Search Engine World site,
and this Sim Spider works properly, you will see that the relevant
links are listed and are spidered properly.

The Search Engine World Sim Spider is here:

When I entered your full URL in this Sim, I got the following results:

As you can see, the relative links are fine, and Google or any other
search engine will have no problem spidering your site.

I hope this helps - I ran across a similar problem using a sim spider,
and the sim spider on the Search Engine World site was recommended.

Search terms -

   * sim spider
   * sim spider + errors


Google Answers Researcher
Subject: Re: Url & Google Spiders: <a href="terms.html">
From: bruce123-ga on 03 Dec 2003 02:07 PST
Ok... as I thought so.

But why when I use the sim spider of for another
website I am working with is it I get the following errors for


The www has beedn dropped... is this another error of THIS sim spider?

Thanks for your help.

Subject: Re: Url & Google Spiders: <a href="terms.html">
From: bruce123-ga on 03 Dec 2003 02:14 PST
Iv'e just realised that has been dropped?

Any ideas.


Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  

Google Home - Answers FAQ - Terms of Service - Privacy Policy