My site www.althealth.co.uk has had its index page spidered twice, but
withing a day google is dropping the site, with google tool bar I
could see that all the front page with its links was visible, and
traffic was comming to the site, but the traffic stopped when google
dropped the site, can you see anything wrong with the codeing, my
programmer has session manager on, but when googlebot appears it is
switched off (just in case this was the problem) do I have to resubmit
the site now google has dropped it? is it the case that if the index
was spidered then the rest of the site would be spidered 2 weeks
later. I am the owner of the site and will pass your answer to my
Programmer  Thanks Mike

Hi Mike,

There was a time when you just submitted your website's URL, waited a
month or two, it appeared and that was that. These days the process
can be more complicated, especially when it comes to Google.
  
Freshbot  
--------  
  
When you submit your URL to Google, some time in the next month or so
the Googlebot will pay your site a visit and index most or all of it.
When the entire main index is refreshed (typically in the first week
of each month) your site should appear and stay there. Sometimes you
have to wait until the following main index refresh for it to appear.
  
Google also has Freshbot - it looks for websites which typically have
constantly changing content, and re-indexes them every day or two. If
you see any Google search results with a date beside them, that is
when Freshbot last visited them. Sometimes Freshbot will
unintentionally find other sites...
  
It appears that althealth.co.uk was found by the FreshBot, and is now
going through an unstable period which often occurs to new sites found
by FreshBot. One of the most common queries I receive here (and
elsewhere) is why does my site disappear, then reappear, then
disappear? The following comes from GoogleGuy, a Google search
engineer who often answers queries posted at WebMasterWorld:
  
GoogleGuy: Everflux and Fresh Crawls  
------------------------------------  
  
If your site is new, or hasn't shown up in google for long, it may
because our "fresh crawl" (which runs each day) was finding your site
instead of our main crawl (which runs about once a month). Our "fresh
crawl" is a newer feature, and we're still experimenting with which
pages to crawl, how deeply to crawl, etc. We even reserve the right to
(gasp!) not do a fresh crawl on some days because we're doing tests or
reviewing new code. Someone wrote in recently and said "my site got in
google three weeks ago, and you've dropped me four times!"  Nope, it's
just that we don't always crawl the same pages in our fresh crawl, and
we don't always crawl to the same depth. As we do a full crawl of the
web, we find most of the sites from our fresh crawl and put them in
our regular index. My advice on our fresh crawl is to view it as a
nice "bonus" on top of google's deep index. Users can always search
our full index, but sometimes we can serve up even fresher pages as an
extra nicety.
  
The rest of the discussion is at:  
http://www.webmasterworld.com/forum3/6339.htm  
  
Basically, if your site is picked up by freshbot one day, but not the
next, the site's listing reverts to its normal listing. If your site
didn't previously have a regular listing, it disappears. Here are some
more opinions at WebMasterWorld:
  
Google also has what we call the fresh-bot, also nicknamed "minty". It
crawls much more frequently, but not as deeply. It is used to update
pages in the index which change often. After a freshbot crawl you site
may appear for a few days with an updated listing, called a "fresh
listing". However, if the freshbot does not return within a few more
days, your "fresh listing" will disappear, and your listing will
revert to whatever it was during the previous deep-crawl, meaning the
one at least one month ago. If you have never had a "previous" deep
crawl, then your site may disappear. If the fresh bot returns to your
site, your fresh listing may re-appear. Therefore, during the first 30
to 59 days of a site's life, it may drop in and out of Google's
results page.
http://www.webmasterworld.com/forum3/6743.htm  
  
When we do a new fresh crawl, it currently replaces the last fresh
crawl. And again, we do experiments from time to time, so different
crawls may visit different pages.
(GoogleGuy)
http://www.webmasterworld.com/forum3/6339.htm  
  
My site is new and isn't in the Main Index yet, but was in for a day,
due to fresh bot crawl 3 days ago. freshbot came back this morning
(whew), grabbed some pages, but I'm nowhere to be found... The point
is that pages found by freshbot have an unsure life in Google's index.
http://www.webmasterworld.com/forum3/5196-3-15.htm  


  
Search Strategy: searching WebMasterWorld for "freshbot"  
  
  
This is a huge subject, with thousands of people guessing at what is
going on, and rare instances of someone like GoogleGuy giving
semi-official answers.
  

Best wishes,  
robertskelton-ga

---

Request for Answer Clarification by mikejwilson-ga on 16 May 2003 16:04 PDT

Thanks its a bit clearer, I can assume my site will be spidered in due
course by one or other of the crawlers, and that my site is not
banned.

Regards Mike

---

Clarification of Answer by robertskelton-ga on 17 May 2003 00:21 PDT

Your assumptions are correct. Your site has definitely not been
banned.

Expect to be spidered sometime this month by the regular GoogleBot,
and to appear in search results when the Google index is next updated,
probably somewhere around June 5-10.