I have a web site http://webpages.charter.net/dfox5/ and the Google
bot came but only picked up one index page. I have other pages in
folders under /dfox5/ but the pages are not named index.htm. These
pages have various names like sears.htm and walmart.htm... Were they
not spidered because they were not named index.htm?

Hello dfox5,

Disclaimer: As a Google Answers Researcher, I am an independent
contractor and not a Google employee.  Thus, I have no inside
information, and cannot give an official reason why the other pages on
your web site were not spidered.

However, I can point to the most likely reasons according to Google's
public guidelines for webmasters.  According to the "Quality
Guidelines" near the bottom of that page:

"Make pages for users, not for search engines. Don't deceive your
users, or present different content to search engines than you display
to users."

"Avoid hidden text or hidden links."

"Avoid 'doorway' pages created just for search engines, or other
'cookie cutter' approaches such as affiliate programs with little or
no original content."

"Google Information for Webmasters: Webmaster Guidelines"
Google
://www.google.com/webmasters/guidelines.html

The most obvious problem is the hidden text in the pages.  For some
reason, this hasn't affected the spidering of the index page.

It is also possible that Google considers the other pages on your site
to be "affiliate programs with little or no original content".

If you decide to make changes to the site, you might want to review
all of the information on the Webmaster Guidelines and related pages,
in order to feel confident that the pages are ones that Google would
approve.

- justaskscott


Search strategy:

Browsed Webmaster information on Google

---

Clarification of Answer by justaskscott-ga on 31 Mar 2004 12:19 PST

Incidentally, hope that you do not infer that I have any personal
criticism of your site.  Rather, my only goal was to compare your site
with Google's Webmaster Guidelines.

I think it's a good answer since it did not come from someone that can
really explain Google. But..... when I search in Google, 3/4 of the
pages that come up are affiliates and most look worse than mine... and
I mean hidden text, etc... not decorations...

Hate to break the bad news to you, but the only one that can explain
Google is Google.  Unfortunately, they're not talking.  Its all
speculation, but welcome to the club. =)