Google Answers Logo
View Question
 
Q: robots, robots.txt, search engine listing in google ( Answered 5 out of 5 stars,   0 Comments )
Question  
Subject: robots, robots.txt, search engine listing in google
Category: Miscellaneous
Asked by: kamy-ga
List Price: $10.00
Posted: 22 Aug 2002 18:24 PDT
Expires: 21 Sep 2002 18:24 PDT
Question ID: 57626
Hello,

My web site address is  www.hippietiger.com.  I received a very
comprehnsive answer as to why I was not listed in google yet. 
Eventhough I have adwords select (spent 370 dollars so far) and I show
up under Internet Marketing and Branding, I am still not lsited in
google. I like to know specificaly why.  If you visit my web site, you
see the content is good and if you look at the source Meta name and
Meta Tags, you see all is good.  Can you verify that.  Is there
something I dont see.  Please.

I will also check with my server to see if there is a crawler block. 
I do not believe so.  My server is Hypermart.net.

I just like to make sure I will be listed.  At one point I was listed
and now I am not.  Please help.

I have listed many of my clients such as Hoferprecision.com and it's
funny I am not there myself.  I also listed Hoferprecision in adwords
select.  I love google and I like to make sure the problem is not on
my side.  Thanks again
Answer  
Subject: Re: robots, robots.txt, search engine listing in google
Answered By: webadept-ga on 23 Aug 2002 11:29 PDT
Rated:5 out of 5 stars
 
Hi, 

Your WhoIs listing shows this information 

   Domain Name: HIPPIETIGER.COM
   Registrar: TUCOWS, INC.
   Whois Server: whois.opensrs.net
   Referral URL: http://www.opensrs.org
   Name Server: NS1.HYPERMART.NET
   Name Server: NS2.HYPERMART.NET
   Updated Date: 24-jul-2002

DomainDuck.com shows your domain on a list of  expired dot com domains
on 06.02.2002 :

FREE EXPIRED DOT-COM DOMAIN NAME LISTS COURTESY OF WWW.DOMAINDUCK.COM
http://www.domainduck.com/html/frames/register/text/06.02.2002-12let-a-z.txt

So apparently this is an expired domain name which you have started up
new again. It may just be that your site has not existed long enough.

Adwords, which I use myself, are small adds on Google which show up
when certain keywords are used to search for a site. The Adword
service has nothing to do with getting listed on Google, nor will it
help in your page ranking. You can not pay for placement on Google.

Quote from Google submission Page :
Google does not accept payment for inclusion of sites in our index,
nor for improving the rank of sites in our results. We do offer
advertising opportunities adjacent to our results, which are always
clearly labeled "Sponsored Links". The method by which we find pages
and rank them as search results is determined by the PageRank
technology developed by our founders, Larry Page and Sergey Brin.

One of the pages this is on is here:
://www.google.com/webmasters/1.html

If you are using Web Position Gold or some other page ranking system,
this can cause you to be removed from the index. Check this website
here.
://www.google.com/webmasters/facts.html

In the same area, a quick list of Do's and Don'ts for Google listings
here:
://www.google.com/webmasters/dos.html

Your site has extensive Front Page extensions on it, in fact it is a
Front Page Template with more stuff thrown in. This may be the cause
of robots or WebCrawler's bouncing off your site. You are not simply
missing from Google, you are also not listed in AllTheWeb.com or
AltaVista.com. So the three main web engines which are currently not
pay-for-placement types, don't have you at all. Extensive usage of
these extensions can make it difficult for a robot to index your page.
Or to get a false reading from it.

You do not have a robots.txt at all, and this will help the indexing
of your site. I would get one on there as soon as you can. You can
find out more about these at this URL:
http://www.robotstxt.org/wc/robots.html

Basically, make a plain text file (don't use MSWord for this, use
Notepad) and put on the top line:
User-agent: *

That's it. Call it robots.txt and place it in the same directory as
your index.htm file. Next make sure that it is readable by going to
http://www.hippietigger.com/robots.txt
If it doesn't pop up or if its not readable, try again using a plain
text file.

You have no page ranking, of course, and searching for pages that have
links to your site shows none found on Google or AltaVista. So once
you are on the index, I would expect that your PR will be low. You can
view your PageRanking by using the Google Bar on your IE browser. You
can find that here:
http://toolbar.google.com/

Google doesn't use the meta tag keywords to list your site. You can
see this by going to
http://www.rule62.com and looking at the source. There are no keywords
there and its still listed, rather well and has a PageRank of 2, which
is pretty good for a site that no one knows exists and isn't updated
very often. Adding keywords to this site would not help nor hider its
placement on Google. For other engines your keywords look fine, I
didn't see any problems with them.

Suggestions:

Your site is a new listing, as far as the Internic is concerned, and
its hard to read from a robots point of view. If I was to work on this
site I would remove all FrontPage extensions from it, and clean up the
code to the basics. Have a white background and a few pictures. Make
sure that all links work to all the pages, and reload it to the
server. Add the robots.txt to the site.

Next I would resubmit the site to Google, AltaVista and AllTheWeb. Do
this by hand and not with a service. Then submit to Yahoo! by visiting
http://docs.yahoo.com/info/suggest/. You can submit your site to
Netscape's Open Directory Project (DMOZ) by visiting www.dmoz.org.
Once your site is included in either of these directories you can
expect to see it appear in the Google index in four to eight weeks,
according to Google's webmaster page.

That should cover the basics and get you listed. 

Thanks, 

webadept-ga
kamy-ga rated this answer:5 out of 5 stars
Thank you
You are great

Comments  
There are no comments at this time.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy