Hi Jsland ~
Since you didn't give your URL in answer to the Request for
Clarification, I will give you the best information I can, which is
general in nature.
All of the information is germane to good search engine results pages
(SERPs), but you may feel some may not pertain to your particular
site. However, if taken as a whole, you can improve your ranking in
SERPs, even if you change content (and you should - you don't want
your site to go stale).
There is a lot of advice out there, much of it bad; but the three key
things every page on your website should have are:
1. Good HTML design;
2. Content, content and more content; and
3. Links TO your site from other, related sites.
Anything else is superfluous. What "may have" worked yesterday cannot
be expected to still work tomorrow, except the above three items, when
applied to a web site - and that means every page within the web site
- aren't adversely affected by any search engine's changes to its
Google's Responsibility to Searchers
It helps to understand Google's responsibilities are to return the
most RELEVANT results to a searcher's query.
In order to do so, Google has indexed about 3.5 billion pages, It is
continually crawling and recrawling the web to update its index.
Add to that the fact that Google, as well as the other major search
engines, continually tweak search engines to better recognize and
deliver relevant pages, and the only thing that can really endure are
those pages which have relative content to a searcher's query.
As recently as last month's Search Engine Session in Chicago, Google
referred attendees to its "Google Today" section, explaining what is
more or less Google's "mission", as well as its guidelines, including
design guidelines and technical guidelines.
"Google Today" is here,
Google's Guidelines can be found here,
The common theme between the two is the relevancy issue. They're
promising the searcher they're always striving to improve to deliver
better, more relevant content; and they're advising the webmasters or
site owners to include relevant material on their websites.
What Google Says
Google's advice is to design your pages for your users/visitors,
"Make pages for users, not for search engines."
and Google even offers guidelines to assist webmasters in
user-friendly and crawler-friendly design. In its "Design and Content
Guidelines" recommends the following:
"* Make a site with a clear hierarchy and text links.
Every page should be reachable from at least one
static text link.
* Offer a site map to your users with links that point
to the important parts of your site. If the site map
is larger than 100 or so links, you may want to break
the site map into separate pages.
* Create a useful, information-rich site and write pages
that clearly and accurately describe your content.
* Think about the words users would type to find your
pages, and make sure that your site actually includes
those words within it.
* Try to use text instead of images to display important
names, content, or links. The Google crawler doesn't
recognize text contained in images.
* Make sure that your TITLE and ALT tags are descriptive
* Check for broken links and correct HTML.
* If you decide to use dynamic pages (i.e., the URL con-
tains a '?' character), be aware that not every search
engine spider crawls dynamic pages as well as static
pages. It helps to keep the parameters short and the
number of them small."
These guidelines aren't new, they've been around for a few years.
Webmasters who follow them and design user-friendly and
crawler-friendly sites always fare better than those who try to figure
out how to game the system for better placement. Gaming may work in
the short-term, but eventually it catches up with those who do it, and
they may be penalized by being completely removed from Google's index.
Links TO Your Site
Google also explains *why* links to your site are important and their
role in SERPs,
"The best way to ensure Google finds your site is for your
page to be linked from lots of pages on other sites.
Google's robots jump from page to page on the Web via
hyperlinks, so the more sites that link to you, the more
likely it is that we'll find you quickly."
And it further explains its technology in "Our Search: Google Technology":
"The heart of our software is PageRank?, a system for
ranking web pages developed by our founders Larry Page
and Sergey Brin at Stanford University. And while we
have dozens of engineers working to improve every aspect
of Google on a daily basis, PageRank continues to provide
the basis for all of our web search tools."
and its PageRank in "PageRank Explained"
"PageRank relies on the uniquely democratic nature of the
web by using its vast link structure as an indicator of
an individual page's value. In essence, Google interprets
a link from page A to page B as a vote, by page A, for
page B. But, Google looks at more than the sheer volume
of votes, or links a page receives; it also analyzes the
page that casts the vote. Votes cast by pages that are
themselves "important" weigh more heavily and help to make
other pages "important."
Important, high-quality sites receive a higher PageRank,
which Google remembers each time it conducts a search. Of
course, important pages mean nothing to you if they don't
match your query. So, Google combines PageRank with
sophisticated text-matching techniques to find pages that
are both important and relevant to your search. Google
goes far beyond the number of times a term appears on a
page and examines all aspects of the page's content (and
the content of the pages linking to it) to determine if
it's a good match for your query."
Without knowing your particular URL, it is impossible to estimate the
type of links to your site. But every link to your site helps,
providing you're not joining some free-for-all or other link scheme.
If you only have a few links, you should work to establish relevant
links from other sites, which would naturally tie in with your own
There are many ways to establish links which are beneficial to you.
These methods may take time, but they also help in establishing
credibility and help with your page rank.
Approach like-minded or complementary businesses about linking to your
site (with a reciprocal link from your own). This works without
harming search engine positioning or page rank.
A WORD OF WARNING:
Google specifically warns "Don't participate in link schemes designed
to increase your site's ranking or PageRank. In particular, avoid
links to web spammers or "bad neighborhoods" on the web as your own
ranking may be affected adversely by those links." (See Google's
Quality Guidelines - Basic principles)
It stands to reason that what's good for Google, currently the number
one search engine, is good rule to follow for other search engines.
Articles on Link Popularity
Some excellent articles on how to establish the right kind of links
are available in Traffick's "Ten Steps to Building Links to Your
Site", Craig Fifield - 5/3/2002
and "The Right Way to Improve Link Popularity", By Paul J. Bruemmer -4/14/2002 -
"Link Building Is Important", which discusses link building from A to
Z and also provides a good reference page with linking resources.
The information contained in the above articles offer suggestions
which can be easily adapted for use on any website without resorting
to link farms. The differences in "good links" and those which may be
harmful are discussed, and the articles offer simple ways to get
started to the kind of linking search engines prefer.
1. Finding Your Site
You asked, "Why was the site not found and what html code should be
used to allow the site to be found now and subsequent to future
As I stated above, without knowing what your site is, it is hard to
answer those questions. The first question that comes to mind is are
you sure you're not listed in Google? Or are you just no longer listed
near the top of your search terms?
To ascertain whether or not you are still in Google's index, have you
actually tried a search on your domain? If it shows up with a listing,
you haven't been dropped from Google's index, although your SERPs
ranking may have dropped.
If it is merely a case of your site having dropped in ranking, you can
pick that back up with good HTML design, rich text content and links
TO your site.
If you think your site has actually been penalized and dropped form
Google's index, some of the answers can be found in its "My web pages
used to be listed and now they aren't."
2. What Your Site Should Contain
You asked in your clarification, "What should be in the home page file
and what should not change for google to find the site reliably?"
You also specifically mentioned "HTML code for the index.htm page can
contain keywords, description, creation date and revision date."
Again, without having your site's URL, I can only answer in the
general nature as I have above.
So far as keywords metatags, they are pretty much ignored, by all
search engines and have for quite some time now. They are good to
have, if for no other than some very small engines may rely on those
keywords metatags to list your site.
They also come in handy to keep you focused on the terms and phrases
which should be contained within the content on your site.
You do not need to load them up with every term you can conceive, but
while it may not help you, it won't hurt to keep focused on each
page's content and which words and terms should appear on each page.
The description metatag sometimes is handy for directories, such as
DMOZ. Try to make your description an entire sentence or two and keep
it somewhere around 250 characters. Think of it as the short snippet
of code you'd want a searcher to see that explains what that page is
Of course, that description would be different for each page,
depending on the content of each page.
c.) Creation and Revision Date
The creation and revision dates may be useful for your own
information, but it really has no bearing on how your site ranks at
It has no effect on how or when the Googlebot or other search engine
crawlers will crawl your site.
d.) Title Tag
You didn't mention the Title Tag (the title metatag which is between
the <head> and </head> tags.
This is your one chance to put a title that others WILL SEE that
should contain the term you want to be found under. Each page can have
its own title tag.
Remember,use it to your advantage, but don't "spam" your title tag,
either. That usually becomes more harmful than helpful.
e.) What should a page contain?
So far as what should be in your home page (and every page, for that
matter) is rich text content. That way every visitor, whether he uses
IE, Netscape, Opera, Mozilla, Firebird, or his cellphone, can view
your page's content. That's what "user-friendly" is all about.
Without your giving us your specific URL (requested in the Request for
Clarification), the answer is general in nature. Make no mistake,
though, it is the same answer I would have given to anyone who
volunteered their URL for close scrutiny. The only difference would be
that I could show you where something might be lacking.
There is no substitute for good HTML design, links TO your site, and
rich, relevant content.
I relied solely on my knowledge of Google and its content guidelines
and other bookmarked resources used in the regular course of my
Best of luck,
Google Answers Researcher