Our site has been www.6sigma.us has been ranking nicely on key words
like "6 Sigma" and "Six Sigma". This past week the site has been
number 2 for "6 sigma" and 12 or 13 on "six sigma". Some days this
past week it was not even ranked.

What is up and what suggestions if any to do you have to resolve. I am
not expecting a book for a complete answer, but some real advice,
suggestions. I have found the other reserachers to always be helpful.

List of Changes:
1. Because of problems and cost last month we changed hosting
companies and our default page was www.6sigma.us/default.asp and now
is www.6sigma.us/default.html  If I type www.6sigma.us I get a
pagerank of 5 and if www.6sigma.us/default.html I get just a pagerank
of 2.

2. In changing providers I needed to purchase other asp programs for
Calendar, eshop. I am also experimenting with a asp links page and
even a asp forum. Most of these are new pages and when I went to
www.marketleap.com we still have all the old pages listed. I did see a
few new ones for one day this past week.

3. I added some new presentations to the site about 100 pages.
www.6sigma.us/SixSigmaProject.html and another
www.6sigma.us/DesignofExperimentsUnit1.html both of these have a
pagerank of zero.

4. I did look at other pages on Six Sigma and the ones that rated
higher for Six Sigma had a higher key word density if 13% for "six
sigma" so I added this term with some image tags and a few other
places. /www.keyworddensity.com/

5. I also changed the homepage title but google does not list the new
title but they do list the new page in their cache with the old title?

Hello peterp,

After quite a lot of reading and a good many interruptions, I believe
I have your situation explained.

I've had a look at your site, and was unable to find anything there
that might cause you to be dropped from the index altogether.  Your
problem is actually quite benign!

You're suffering the effects of both changing your URL in the past
month, and the current Google Dance.  Neither of these are anything to
worry about, they'll shake themselves out in due course.

First, let's explain the Google Dance - the affectionate nickname
given to the period each month in which Google updates the index. 
During the Google Dance, page rankings and standings can shift wildly
or disappear altogether in a matter of days, hours or even minutes as
new pages found are added, dead links are discarded, and content
changes are considered.

James Kendall at SEO Today explains the Google Dance: 
 
"The Google Dance 
 
Google has at least four different indexes that are manipulated to
test different results when the Google Dance is on.  During the dance
you can get a glimpse of how your site ranks in their different
indexes.  If you do a search in the default Google index around the
time of an update, and then do the same search in one of the other
indexes, you will get different results.  For example, on February 22
you could have gone to http://www2.google.com/ to preview the new
Google index and compared the current listing (www.google.com) to next
month's (www2.google.com).
 
The time surrounding Google's updates is normally referred to as the
Google Dance because the databases are switched around and back a
couple of times before things become stable."
 
It's All About Google - SEO Today, February 26, 2002 
http://www.seotoday.com/browse.php/category/articles/id/173/index.php

"What is the Google Dance?
Once a month, and totally unanounced,  Google has a major shift in
it's rankings.  This is when Google "tweaks" is algorithm, and when it
updates each sites PageRank and Back Links.

During the month there will be minor changes in rankings.  This is
called 'Everflux'.  But only about once per month does Google Dance,
updating the back links and the PageRank.  The dance usually lasts
about 3-5 days.  During these days the Google Results will vary
widely."

The Google Dance Explained
http://www.internet-advertising-marketing-manual.com/google-dance.htm

"The best time to put new pages online is during the Google-Dance. If
you let too much time lapse between the completion of the update, and
the publication of new content, you may reduce the amount of new
content that will be included in the next update."
 
Google Update - The Google Dance 
http://www.linktree.info/googleupdate.php 

"The name "Google Dance" is often used to describe the index update of
the Google search engine. Google's index update occurs on average once
per month. It can be identified by significant movement in search
results and especially by Google's cache of all indexed pages
reflecting the status of Google's last spidering. But the update does
not proceed as a switch from one index to another at one point in
time. In fact, it takes several days to complete the index update.
During this period, the old and the new index alternate on
www.google.com. At an early stage, the results from the new index
occur sporadically. But later on, they appear more frequently. Google
dances."

Google Dance - The Index Update of the Google Search Engine
http://dance.efactory.de/

There's even a nifty cool tool, called the Google Dance Machine, that
lets you see if Google is indeed doing the WWW Boogie:

Google Dance Machine
http://google-dance.miniunternehmen.de/

This is why you're seeing such wildly changing results - Google has
its dancing shoes on and is sorting out and building the new index. 
This is completely normal (if a bit nerve-wracking for the
unsuspecting webmaster!), and occurs approximately every four weeks.

Now let's address the rest of your query.

The reason your new pages aren't showing up in the index yet is
because...well, they're new!  The Googlebot hasn't found your new
links yet, so your new pages aren't going to show up in the index
until it does.

Here's how it works, starting with Page Rank, and progressing through
getting indexed and what happens when you change your URL:

" PageRank relies on the uniquely democratic nature of the web by
using its vast link structure as an indicator of an individual page's
value. In essence, Google interprets a link from page A to page B as a
vote, by page A, for page B. But, Google looks at more than the sheer
volume of votes, or links a page receives; it also analyzes the page
that casts the vote. Votes cast by pages that are themselves
"important" weigh more heavily and help to make other pages
"important."

Important, high-quality sites receive a higher PageRank, which Google
remembers each time it conducts a search. Of course, important pages
mean nothing to you if they don't match your query. So, Google
combines PageRank with sophisticated text-matching techniques to find
pages that are both important and relevant to your search. Google goes
far beyond the number of times a term appears on a page and examines
all aspects of the page's content (and the content of the pages
linking to it) to determine if it's a good match for your query."

PageRank Explained
://www.google.com/technology/index.html

"We update our index every four weeks. Each time we update our
database of web pages, our index invariably shifts: We find new sites,
we lose some sites, and sites ranking may change. Your rank naturally
will be affected by changes in the ranking of other sites. You can be
assured that no one at Google has hand adjusted the results to boost
the ranking of a site. Google's order of results is automatically
determined by several factors, including our PageRank algorithm.
Please check out our "Why Use Google" page for more information on how
this works.

    You may want to check and see if the number of other sites linking
to your URL has changed. This is the single biggest factor in
determining what sites are indexed by Google, as we find most pages
when our robots crawl the web and jump from page to page via
hyperlinks. To find out who links to your site, use Google's link:
tool."

Google Information for Webmasters - Why does my page's rank keep
changing?
://www.google.com/webmasters/4.html

"Google finds sites through a process known as "crawling" the web. 
This involves robot software that follows hyperlinks from site to 
site. Google currently looks at more than 3 billion URL's during the 
crawl. The process may take several weeks to complete. 
  
When a URL is submitted to Google, we look for it in our next crawl. 
If you've already submitted your URL, your site could easily appear in
our new index, which will go up when the current crawl is completed. 
However, if no other site links to yours, it may be difficult for our
crawler to find you. Conversely, if many sites link to your page, 
there is a good chance we will find you without your submitting your 
URL." 
  
[...]  
  
"If we have not picked up your site and it has been several months, 
then it is likely that our spiders are not able to find your site. If
you increase the links pointing to the page, Google will likely find 
your site in the future." 
  
Getting Listed  
://www.google.com/webmasters/1.html#B1  

"We cannot manually change your listed address at the exact time you
move to your new site. There are steps you can take to make sure that
your transition goes smoothly, however. Google listings are based in
part on our ability to find your site by following links from other
web pages. To preserve your ranking, you will want to inform any sites
that currently link to your pages of your change of address. As long
as the links change as you move your site over to a new location, your
PageRank should not be adversely affected.

If your site goes unlisted for a time, this does not mean you were
intentionally dropped from our index. Sometimes in these transitions,
we fail to find a site at its new address. Just be sure that others
are linking to you and we should pick you up on our next web crawl."

Google Information for Webmasters - I'm Changing My URL
://www.google.com/webmasters/3.html

Notice the mention of links in the last section.  Leaving aside the
fact that your new URL is new, your new URL doesn't have anyone
linking to it yet.

Have a look:

http://www.6sigma.us/default.asp shows 116 links pointing to it:

Google Search Results: link:http://www.6sigma.us/default.asp
://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&newwindow=1&q=link%3AOrYKbOWQLGkJ%3Awww.6sigma.us%2Fdefault.asp

http://www.6sigma.us/default.html, on the other hand, shows no links
pointing to it yet:

Google Search Results:  link:http://www.6sigma.us/default.html
://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&newwindow=1&q=link:AdjYaB0WUOkJ:www.6sigma.us/default.html

http://www.6sigma.us reveals 116 links pointing back to it:

Google Search Results:  link:http://www.6sigma.us
://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&newwindow=1&q=link:OrYKbOWQLGkJ:www.6sigma.us/

At the moment, your best option is simply patience and "change of
address forms" - let those who've linked to the old page know that
you've changed your URL and ask them to update their links accordingly
so the Googlebot will be able to find your pages more easily on its
next crawl.

In the meantime, issue the Googlebot an invitation for a visit -
re-submit your URL here:

For Site Owners: Submit your site 
://www.google.com/addurl.html

...and hang in there.  Once the Googlebot has the current links to
jump to, you should see an improvement as early as the next Google
Dance (about 4 weeks hence).

While you're waiting for things to sort themselves out - and they
will! - why not have a look at some of the resources available for
webmasters to help them stay abreast of changes and tweaks:

Google Information for Webmasters    
://www.google.com/webmasters/    
    
Google Webmasters' FAQ    
://www.google.com/webmasters/faq.html    
    
User Support Forum    
http://groups.google.com/groups?q=google.public.support.general 

I hope this helps ease your mind a bit!  If I can be of further
assistance, please don't hesitate to ask for clarification.  I'll be
glad to help you!

--Missy

Search terms:  None.  Bookmarks and Google FAQs.

---

Request for Answer Clarification by peterp-ga on 15 Jun 2003 02:30 PDT

Missy,


That for the detail effort. I am confused. I never asked any one to
link to the default page of http://www.6sigma.us/default.asp or
http://www.6sigma.us/default.html all my links go to
http://www.6sigma.us. So what is up on this? How is google treating
the default home page? My new provider simply uses the default.hmtl as
the home page. Would this also occurred if I changed to index.html.

I keep doing test searches and at time I get a cached file of my new
first page and the date is June 13th. Does this mean my site is back
into google?

Thanks

Peter

---

Request for Answer Clarification by peterp-ga on 15 Jun 2003 03:00 PDT

MIssy,

I did some searches and this came up
http://www.webmasterworld.com/forum3/12211.htm

They have a discussion about the same idea of changing default pages.
I still don't have clear direction. They seem to indicate to use 301
or 302. This may be what I need to do. What is it? I have used
robot.txt file to exclude another duplicate site from google.

Is this something I need to do?

Thanks for the clarification.

Peter

---

Clarification of Answer by missy-ga on 15 Jun 2003 11:01 PDT

Hi Peter,

At this point, it's too early to say if you need to include
re-directs.  You *just* changed your URL a month ago, and from all
appearances, not in time for the Googlebot to pick those changes up
for the most recent crawl.

The standard implementation of a 301 permanent redirect (the web
equivalent of a "We've Moved!" sign) is through your .htaccess file:

"How do I implement a 301 redirect?

First of all, you'll need to download the .htaccess file in the root
directory of where all your web pages are stored. If there is no
.htacess file there, you can create one with Notepad or a similar
application. Make sure when you name the file that you remember to put
the "." at the beginning of the file name. This file has no tail
extension.

If there is a .htaccess file already in existence with lines of code
present, be very careful not to change any existing line unless you
are familiar with the functions of the file.

Scroll down past all the existing code, leave a line space, then
create a new line that follows this example:

redirect 301 /old/old.htm http://www.you.com/new.htm 

It's as easy as that. Save the file, upload it back into your web and
test it out by typing in the old address to the page you've changed.
You should be instantly and seamlessly transported to the new
location.

Notes: Be sure not to add "http://www" to the first part of the
statement - just put the path from the top level of your site to the
page. Also ensure that you leave a single space between these
elements:

redirect 301 (the instruction that the page has moved)

/old/old.htm (the original path and file name)

http://www.you.com/new.htm  (new path and file name)

Search engine spiders & 301 redirects

The 301 redirect is the safest way to preserve your rankings. On the
next spidering, the search engine robot will obey the rule indicated
in your .htaccess file. The search engine spider doesn't actually read
the .htaccess file, but recognizes the response from the server as
valid. In the next update, the old file name and path will be dropped
and replaced with the new one. Most importantly, the 301 redirect is
recognized by the mightiest of search engines - Google."

Tutorial - Giving search engine spiders direction with a 301 redirect
http://www.tamingthebeast.net/articles3/spiders-301-redirect.htm

You can also do it in .ASP using the instructions offered here:

301 Redirect on Microsoft Server - What is the equivalent?
http://www.webmasterworld.com/forum47/344.htm

Keep in mind that it can take a month or more for pages to be "found"
by the Googlebot, so even if you were to implement a 301 redirect
today, you'd still have to wait at least four weeks for the next crawl
for the new results to show up.  The index updates only once a month.

--Missy

---

Clarification of Answer by missy-ga on 15 Jun 2003 11:16 PDT

Hi again,

I see I missed a bit - I keep getting interrupted in my work!

You ask:

"I keep doing test searches and at time I get a cached file of my new
first page and the date is June 13th. Does this mean my site is back
into google?"

Most likely yes.  Remember, Google is dancing at the moment, so you're
going to get inconsistent results, depending on which server you hit. 
You will continue to get inconsistent results  - sometimes your new
page will show up, sometimes it won't - until the Google Dance is
completed. (Refer to the pages above explaining the Google Dance in
detail.)

With respect to your backwards links, you might also want to review
Google's guidelines:

How Do I Get My Site Listed on Google?
://www.google.com/webmasters/1.html 

...for a clearer understanding of how the Googlebot goes about its
business.

Your site is fine, you don't appear to have been penalized in any way,
you're simply caught in the current update cycle.

--M

---

Request for Answer Clarification by peterp-ga on 15 Jun 2003 11:32 PDT

Missy,

Almost satified. You mentioned that google saw all the links to
www.6sigma.us/default.asp when in reality I found only two sites that
used this link. Essentially it appears that Google treats the old
default page www.6sigma.us/default.asp and www.6sigma.us as one in the
same. In marketleap only Google shows 150 links to
www.6sigma.us/default.asp. All other search engines just two.

Based upon the data this appears to be same problem that occurred with
http://www.webmasterworld.com/forum3/12211.htm

Any idea how to get google to look only at www.6sigma.us and disregard
the actual default page name? Lastly any way to tell google that the
defualt page is different?

Thanks 

Peter

---

Clarification of Answer by missy-ga on 15 Jun 2003 15:40 PDT

Hello Peter,

You ask:

"Any idea how to get google to look only at www.6sigma.us and
disregard
the actual default page name?"

The Googlebot reads links *as they are written*, and follows from link
to link.  Inspecting your internal links reveals that all of your
"Home Page" links point to http://www.6sigma.us/default.html .  Change
those to http://www.6sigma.us/ .

This won't guarantee that the Googlebot will ignore the actual page
name, but it will help.

"Lastly any way to tell google that the defualt page is different?"

You can re-submit your URL here:
 
For Site Owners: Submit your site  
://www.google.com/addurl.html 

...but it will not speed up the inclusion of changes to your site. 
The index is updated only once a month, so whatever is in the index
when the Google Dance finishes is what will remain there until the
next crawl in four weeks.

--M

Missy did a good job taking the information and with interaction
getting to a solid answer. The question and answer has many unknowns
since Google does not have a clear method (at list from an outside
view) or handling page name changes.

Thanks

This was an excellent, thorough, and comprehensive answer. It is a
shame that the questioner felt it necessary to downrate the Researcher
because of dissatisfaction with a policy of Google itself, over which
the Google Answers Researchers obviously have no control.

I rated the answer. I felt the answer was a three out of five that is
my option.

It is a difficult area with many unknowns and I am sure many
researchers passed up the question. I did feel the answer was worth
what I paid for it. So that should be satification of a job well done
by the researcher.

I would have rated the reseacher higher but the researcher missed the
fact that I had only two external links going to the .asp home page.
This was the root of my problem and for some reason Google sees
external links going to 6sigma.us/default.asp when they actaully do
not point to this page. They point just to the 6sigma.us. This is
still unknown as to why this happens. Other people have had the same
problem as I referenced the the link
http://www.webmasterworld.com/forum3/12211.htm I did get a solution
that I could use the future after several rounds of clarifications.

I did not downrate the rating because the information is not released
by Google or any dissatification I may or may not have in Googles
policies.

I certainly wish the researcher a great day and thank you.

Aw, magnesium, that was really sweet of you to jump to my
defense...but really not necessary.

I have a personal philosophy about ratings:

http://answers.google.com/answers/main?cmd=threadview&id=175507

At the end of the day, what matters is whether or not the customer is
satisfied with the work.  He's not hollering for my head on a plate or
calling me incompetent, so I'm satisfied that I did my best to give
him a good answer based on the information available to me.  It
appears that Peter was as well, and that's fine with me.

--Missy