Google Answers Logo
View Question
 
Q: Url & Google Spiders: <a href="terms.html"> ( Answered,   2 Comments )
Question  
Subject: Url & Google Spiders: <a href="terms.html">
Category: Computers > Algorithms
Asked by: websitepromotions-ga
List Price: $30.00
Posted: 27 Nov 2003 09:14 PST
Expires: 27 Dec 2003 09:14 PST
Question ID: 281174
Enquiry:

Is it important to have the full URL i.e.
http://www.bothalkennels.co.uk/index.html
Within the html code of the page so the search engines are able to
identify the link and crawl to the next page successfully?

I sometimes use a script driven navigation menu (mapped - by Microsoft
FrontPage) which when coded is like this <a href="terms.html"> and is
also displayed as the same when viewing the source code from the
online web page but returns results like the urls listed below when I
use a sim spider which is of course incorrect. Is this an error with
the sim spider or is it because <a href="terms.html"> is not
negotiable by the search engine spiders including Google.

(The / slash is missing after .co.uk)

   http://www.bothalkennels.co.ukindex.html 
   http://www.bothalkennels.co.ukbookings.html 
   http://www.bothalkennels.co.ukkennels.html 
   http://www.bothalkennels.co.ukcontact.html 
   http://www.bothalkennels.co.uklocation.html 
   http://www.bothalkennels.co.ukdocumentation.html 
   http://www.bothalkennels.co.ukterms.html 


These should be: http://www.bothalkennels.co.uk/index.html  

Hope someone can clarify this.

Bruce
Answer  
Subject: Re: Url & Google Spiders: <a href="terms.html">
Answered By: serenata-ga on 02 Dec 2003 18:22 PST
 
Hi Bruce ~

To answer your question about links and a sim spider --

relative links (which are links that do not contain the full URL but
just the individual page in relation to where they are placed) are
spidered just as well as absolute links (which are links containing
the full URL of a page).

I have a feeling you may have used a sim spider which contains errors.
I found one on the WebDevStore website, here:
   - http://www.webdevstore.com/tools/simspider/

whatever happens, it does produce an error when using "co.uk" in the
URL, although it doesn't seem to have problems with the "regular" top
level domains (ie., .com, .net, etc.) Maybe it's the two periods
because of ".co.uk"

Previously, problems have been noted with certain sim spiders when
using relative links. For some reason, certain sim spiders do not
always realistically simulate how Google - or any other search engine
- would read the links, because Google and other search engines can
find and index relative links with no problems.

If you think about it - server side includes (SSI) and cascading
stylesheets (CSS) are fed via relative links; and if search engines or
browsers couldn't read relative links, they would not be able to
deliver your content as you want it seen.


I located a discussion from Webmaster World on just that subject -
errors in sim spider simulators - in a thread entitled "Sim Spider not
finding links - not finding internal links":
   - http://www.webmasterworld.com/forum19/988.htm


If you use the Sim Spider available on the Search Engine World site,
and this Sim Spider works properly, you will see that the relevant
links are listed and are spidered properly.

The Search Engine World Sim Spider is here:
   - http://www.searchengineworld.com/cgi-bin/sim_spider.cgi


When I entered your full URL in this Sim, I got the following results:

  http://www.bothalkennels.co.uk/index.html
  http://www.bothalkennels.co.uk/index.html
  http://www.bothalkennels.co.uk/bookings.html
  http://www.bothalkennels.co.uk/kennels.html
  http://www.bothalkennels.co.uk/contact.html
  http://www.bothalkennels.co.uk/location.html
  http://www.bothalkennels.co.uk/documentation.html
  http://www.bothalkennels.co.uk/terms.html


As you can see, the relative links are fine, and Google or any other
search engine will have no problem spidering your site.

I hope this helps - I ran across a similar problem using a sim spider,
and the sim spider on the Search Engine World site was recommended.


Search terms -

   * sim spider
   * sim spider + errors
   * www.bothalkennels.co.uk


Regards,

Serenata
Google Answers Researcher
Comments  
Subject: Re: Url & Google Spiders: <a href="terms.html">
From: bruce123-ga on 03 Dec 2003 02:07 PST
 
Ok... as I thought so.

But why when I use the sim spider of
http://www.searchengineworld.com/cgi-bin/sim_spider.cgi for another
website I am working with is it I get the following errors for
www.scot-gift.com

    http://main.asp?sitepages=HomePage/ 
    http://viewcart.asp/ 
    http://main.asp?sitepages=competition/ 
    http://main.asp?sitepages=freerecipes/ 

The www has beedn dropped... is this another error of THIS sim spider?

Thanks for your help.

Bruce
Subject: Re: Url & Google Spiders: <a href="terms.html">
From: bruce123-ga on 03 Dec 2003 02:14 PST
 
Iv'e just realised that www.scot-gift.com/ has been dropped?

Any ideas.

Bruce

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy