Please help me! There is some problem causing my site NOT being in
your index that does not fall into the standard boiler plate solutions
provided by help@Google.com [#1510642].
My site is a non-profit, non-commercial site dedicated to recognizing
the "Greatest Generation" (WWII and the Korean War.) It was started in
the mid 90s. It had an excellent position in Google for a several
years. Now it isn't there at all, although hundreds of other sites
link to it. In fact, if you search Google for "Kilroywashere" most of
the sites in the first few pages are there because they link to my
site.
1. It was in Google with a good page rank until about a year ago.
2. I have spent a year submitting it every month so it could not be a
temporary glitch.
3. It is NOT now in your index although I have removed the frames and
did everything else to make it more readable to your spider. Proof
that it is not in your index is:
a. When I checked the links to my page
(link:http://www.kilroywashere.org) again, IT FOUND NONE even though
there are hundreds of sites linking to it! The last time I tried this
(in 2001) there were 288 reported by Google. For some reason, Google
is not recognizing links to www.kilroywashere.org.
Lycos uses a similar method to find links. It still shows almost
200. Google itself, if you search for "kilroywashere", reports 53
sites in the first 13 pages that are listed there only because they
link to www.kilroywashere.org but Google does not list
www.kilroywashere.org AT ALL. I'm not talking about page ranking but
a complete absence of a listing!
Additionally, starting on page 2 of the search is a
www.Kudzumonthly.com that has only 124 sites linking to it. Also on
page 2 is where Zeal links to my site (I'm well represented on Zeal.)
I spent several hours looking at these links. Some are popular sites.
Some are obscure sub pages within a site but they ARE THERE.
b. Your own "Free Web Search" plus declines to allow me to join
because it says my site is not in your index.
4. I have subscribed to Alta Vista (infospider,) Ask Jeeves
(Ineedhits,) (Inktomi,) and Lycos (InsiteSelect,) and they are all
able to spider the site.
All those listed in item 4 do not make up for not being listed in
Google. Can you please help me figure out why www.KilroyWasHere.org is
not listed at all in Google nor does Google recognise any links to it.
Request for Question Clarification by robertskelton-ga on 14 Feb 2003
14:18 PST
Have you ever used software like WebPosition Gold to check your
ranking in search results?
Request for Question Clarification by sgtcory-ga on 14 Feb 2003 19:07
PST
Hello Kilroy -
Firstly - Semper Fi for the great content at the site!
Google Answers is a seperate entity of Google, and does not control
the listings within the Google index. However - I can offer you
insight as to some of the common misconceptions of getting listed ,
and quite possibly help with your site listing problems should you
desire. I do think this may relate to the popularity of the sites that
may link yours, and not the quantity.
Please let me know if I may assist you further, and I will do my best
to help to figure out your listing issues.
Look forward to your reply -
SgtCory
Clarification of Question by kilroy-ga on 15 Feb 2003 09:09 PST
To robertskelton-ga
Thanks for the response. No, I tried one a couple of years ago called
"Spider Submit" but it didn't seem to do anything except get me
literally hundreds of spams that never stopped. I checked the
rankings manually but ranking is not my problem - just getting back in
the index is. I figured I got "Blacklisted" but Help@Google.com
assured me that I wasn't. I did subscibe to ABS Scrub The Web and did
everything they suggested for a year to no avail.
Clarification of Question by kilroy-ga on 15 Feb 2003 09:21 PST
To sgtcory-ga
Thanks for the response Sarg and Sempre Fi! How did you know I was
jarhead? Must have been the intelligent, articulate question. Thanks
for the kind words about the site too. I will gratefully accept any
insight you have. I feel a responsibilty to to vets that trusted me
with their stories.
Like I said above, I subscibed to experts at ABS Scrub the Web and
followed all their suggestions for a year but with no help. I can't
help but suspect it is a problem at Google that I have no control over
since they show no links to my site (as theu used to) although search
results find many. Their own "Free Web Search" can't find my site.
Request for Question Clarification by sgtcory-ga on 15 Feb 2003 14:30
PST
kilroy -
Great! I will offer you some insight first thing in the morning. (Feb
16th EST)
I myself am a jarhead, so I assume we all have some sort of *gut
feeling* way of knowing these things :-)
I have answered many questions with regards to Google, and have helped
many webmasters with my honest, rule abiding approach to getting
listed. I am sure I will be able to answer your questions in a manner
that will provide you with results.
Thanks for the opportunity!
SgtCory
Clarification of Question by kilroy-ga on 15 Feb 2003 18:53 PST
To kemlo-ga
Thank you for the kind words! As you probably know, we really
appreciate any feed back. What do you mean "played a little faster"?
You mean the tempo or should it start faster?
Answer
There is no answer at this time.
Comments
Subject: Re: Web site dropped by Google
From: serenata-ga on 14 Feb 2003 19:55 PST
Hello Kilroy -
Google Answers Researchers are independent contractors, and we are not
privy wo inside and proprietary information on Google's practices.
I can direct you to Google's Webmaster Information page which contains
some FAQs that may be helpful:
://www.google.com/webmasters/2.html#A1
I can also direct you to Dany Sullivan's Search Engine Watch at:
http://searchenginewatch.com/
Search Engine Watch is generally acknowledged as having valid and
helpful information.
Other "common sense" items to watch for are statements which may be
misleading, etc., and I have no idea how many of these information
sites you have taken into consideration.
If anyone presumes to guarantee your inclusion in any search engine,
you should run away as fast as you can, as no one, even the 'experts'
is really privy to the algorithms and other criteria used by the
search engines. The best anyone can do is make a really good guess or
even an educated guess on how to get you included and rated.
So far as guesses go, I would question your use of the tld .org in
your name. That is generally used by 501(c)(3) non-profit
organizations, and if you don't qualify as such, I'd wonder at the use
or .org. It "could" be something as simple as that. It could also be
any one or number of other things that prevent you from being listed,
so don't place any premium on what I said about using .org, either.
There is an email address on Google's Webmaster's guide and you might
try contacting them to see if they can shed any light on the
situation.
I know this is no help, except perhaps as a caveat.
Yours ever so,
Serenata
Subject: Re: Web site dropped by Google
From: kemlo-ga on 15 Feb 2003 17:41 PST
Dear Kilroy
Love the site. Just wish the music played a little faster. Keep it
up/on.
Subject: Re: Web site dropped by Google
From: sgtcory-ga on 16 Feb 2003 07:18 PST
Good Morning kilroy,
As I mentioned, I am offering this as means to help you resolve some
of the most common issues with getting your site listed in the Google
index. I can not guarantee a listing, and no other company can either.
I do know of many things that may need to be looked at, and they will
most definitely help get you listed.
Let's start with the most obvious -
1) Stop submitting your site to Google on a monthly basis. Although
Google will list sites in this manner, the Page Rank feature of Google
plays an important role. If you list your site using the submission
tool, but Google can not find sites with good ranking linking to you
and already in the index - this approach may prove fruitless. (And
even do more harm than good)
I would take the *start-fresh* approach, and find sites that
compliment yours. Once you find relevant sites that easily place your
site into a topical category, you will find it easier to get your site
classified. Here are a few sites that I found, that you may want to
contact for a link exchange -
Korean War Webs
http://www.kimsoft.com/kr-war.htm
WWII Links on the Net
http://history.acusd.edu/gen/ww2_links.html
Korean War Project
http://www.koreanwar.org/com/add_url/entryform.cfm
2) Contact the owner of the Kilroywashere.com site and ask for a
'gratis' link. The site has a good listing, and visitors may well be
looking for your site. I would approach them and ask for a link that
asks visitors if they are looking for the Kilroywashere.org military
related site. The site name detracts from your listing, and may
confuse/alter some of your listings.
Your site is already well placed within Yahoo and the Open Directory.
The fact that your site is not listed in Google is quite perplexing.
It's easy to see though, that your site is not forgotten, but likely
not listed due to one of the factors I have, and will list here. Your
site can still be found in the Google directory, proving that your
site is afforded the opportunity to be listed. You can see your site
in the Google directory here :
http://directory.google.com/Top/Society/History/By_Time_Period/Twentieth_Century/Wars_and_Conflicts/
Your site also shows up in a search for the DMOZ description of your
site. It is about number 7 :
://www.google.com/search?q=WWII+%26+Korean+war+sites+and+stories
This really leads us into a Page Rank issue. Finding more sites like
the ones I listed above may well be the only problem. Since your site
has been around for considerable time, many of the sites that used to
link to yours may have also been dropped from the Google index,
creating an even lower Page Rank for your site.
3) Check your server logs and statistics for accesses. Where is this
traffic coming from? If your site does get indexed by Google, the
Googlebot will appear in your server logs. Installing a small tracking
program using PHP or Perl will help you greatly in detemining where
your traffic is coming from. You will be able to see what other sites
in the Google index are sending you traffic, and how relevant this
traffic is. You can then make adjustments as needed off of the
results.
4) I don't believe this is an issue with your TLD. (.org) Google does
not usually concern themselves with such issues, as this is done at
the registrar level.
Without much more digging, I can place my expertise in this area on
the Page Rank.
I hope these tips help you a bit more, and good luck with your site! I
did not offer this as an answer due to the nature of the question. If
you find you like the results of this answer, I can offer it as such
and give many more tips to help you get back to where you once were.
Thanks again and Semper Fi!
SgtCory
Subject: Re: Web site dropped by Google
From: kemlo-ga on 16 Feb 2003 08:25 PST
Dear Kilroy
Please accept my sincere apologies, I did not realise the tune is your
country's National Anthem. The site is still very good.
Subject: Re: Web site dropped by Google
From: kilroy-ga on 18 Feb 2003 07:00 PST
To kemlo-ga
No problem! Thank you for your input anyway. You can't be expected
to recognize everyone's National Anthem. Search the site (search box
on bottom of main pages.) There are stories and letters from all over
the world. Especially those involved in WWII (both sides.) Your
country may be represented. If not, ask any old timers you know if
they remember something that should be here. The memories must not be
forgotten!
Subject: Re: Web site dropped by Google
From: ss64-ga on 22 Feb 2003 03:26 PST
The answer to this problem can be found in your question...
>1. It was in Google with a good page rank until about a year ago.
>3. It is NOT now in your index although I have removed the frames
and
>did everything else to make it more readable to your spider.
This tells us two things, first the site has content that's good
enough to get listed and second that since making a major design
change you have lost the listing.
As others have commented it's possible your site has just lost
popularity over the last year - this seems unlikely given the time
period you cover, it's also possible that the google algorithm has
changed reducing your ranking, however I think it's far more likely
that changing the design has mucked things up.
A quick look at the source code reveals that all your links are coded
with JavaScript e.g.
onClick="MM_openBrWindow('001-Pages/1-Menu-1.htm',
The GoogleBot is not a conventional Web Browser - it ignores all
javascript; just like it ignores all graphics - it will only follow
links defined in HTML as "href="
e.g. href="http://www.kimsoft.com/kr-war.htm"
So while I would agree that getting rid of frames is a good thing, you
also need to add some conventional links for the GoogleBot to follow.
From the Google Webmaster Info page:
"If your site's internal link structure does not provide a path to all
your pages, our robot may not see all the pages on your site."
In your case the robot won't see ANY pages other than the home page.
A popular way of fixing this (that won't involve removing all the
javascript) is simply to add a site Map - create one page with "href="
links to every other page in the site and then add an
"href=sitemap.html somewhere on your home page.
Also make sure that every page has a link back to the home page -
because the WebBot may start it's search on any page.
After making this change allow up to a month for the GoogleBot to
visit - look for something like this in your referrer logs:
crawler12.googlebot.com - - [19/Feb/2003:20:12:21 -0500] "GET
/robots.txt
Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
After the Bot has visited allow another month before the results
actually appear in Google.
Continually submitting your site to Google definitely won't do you any
favours, but I don't think it would be enough to get you delisted.
Finally - try loading the site in some alternative web browsers
(Netscape and Mozilla) I think you'll find some more work to do there
- I found it almost impossible to navigate in Mozilla.
Hope this helps
Subject: Re: Web site dropped by Google
From: kilroy-ga on 24 Feb 2003 10:14 PST
To sgtcory-ga
Sarg, I really appreciate the specific answers. I am doing as you
suggested. I have a links page which I call the Volume 8 Research
page at:
http://www.kilroywashere.org/008-pages/08-0Research-Links.html
where, at one time, every one linked back to me. I’m going through
each to determine if they really do and, if not I ask them to do so.
The site who’s name is almost identical, www.kilroywashere.com even
sent me a story from his father that is featured on the first page
“Yorliks Me109." I made your suggestion anbout mutual links but he
hasn’t responded. I don’t believe that will help a lot though since
he only has a total of only a little over 24,000 hits and I have more
than 10 times that much (approaching 400,000.)
Your suggestions are great and I appreciate them enough that they are
good enough to be an “Answer” (however THAT works) but they still
don’t address the most vital problem. Rank is NOT the problem, just
getting listed is. It is, indeed “perplexing.” Google doesn’t
recognize my site nor ANY sites linking to me although their own
search finds many linking to me when you search for kilroywashere.
After I get listed at ANY rank, I can start working on rank again.
Like I said, I used to be second or third.
Web Pro News suggested I call them and gave me a phone number. I will
give this a little more time and do that. Thank you again! Sempre
Fi!
Subject: Re: Web site dropped by Google
From: sgtcory-ga on 24 Feb 2003 10:37 PST
Glad you liked the tips I offered.
I have been doing this type of work for a long time. If the problem
here is not affiliated with Page Rank I would be willing to do 1000
pushups. (In broken up intervals of 5 :-)
As perplexing as it may seem at times, the concept is really simple:
- List sites that other sites think are popular.
- Reward sites that make an effort to *maintain* a good *web* status
It may just be that some of the sites that link to you are not as
*fruitful* as they once were.
A few questions and I might be able to compile all of this and get you
a very valid answer:
1) Do you know what robots.txt is? If so - do you use one/it?
2) Sitemap page - was this indexed before?
I suggest doing just as you stated, and wait it out. I am positive
your site will be indexed with the next 2 months. I'll let you know
how I know in our next conversation - which will hopefully lead to a
full answer to your questions.
Thanks and keep up the good service to country!
SgtCory
Subject: Re: Web site dropped by Google
From: kilroy-ga on 24 Feb 2003 10:46 PST
To ss64-ga
First, let me thank you for help! I believe you hit on a problem that
was troubling me! Even if it isn’t the answer to my specific
problems, it is certainly worth being and “Answer” even if not the
answer I am looking for – how does that work?. I didn’t know about
the Java problem and Google. It seems I have spent more time removing
fancy stuff than putting stuff in. Until I can figure out a better
way of navigating, I am taking your suggestion and adding a Site Map
for Google. If you look, you can see that it is IMMENSE! It is not
finished but I put a lot there so I don’t miss another Google “Dance.”
I is a huge job and will have a lot of maintenance but worth it if it
allows Google to search down into the site. Recently, I was asked to
write a piece for an anthology. It was amazing! When it was
published and put on a web site, withing weeks my name was listed on
the search engines even though it was several pages deep withing the
site. I never could achieve that – now I know why! This seems to
explain why pages deep within Kilroy are not found but not why page
one is not there. As you said, “In your case the robot won't see ANY
pages other than the home page.” Google doesn’t see my home page
either! Which is the main problem.
As to the other points. It hasn’t lost any popularity. Google just
doesn’t see it any more. Google doesn’t recognize my site nor ANY
sites linking to me although their own search finds many linking to me
when you search for kilroywashere. I have been laboriously checking
and it is still linked to hundreds of sites. Lycos finds many sites
linking to me but Google doesn’t.
As to loading in other web Browsers, I use Netscape 7 (which is now
Mozilla I think) as my primary browser. The only thing I have noticed
between it, Netscape 4.79 and IE 6 is that I am unable to force the
little menu pages to stay on top in Netscape as it does in IE 6.
Please be more specific about the nav problems? When I get the Site
Map finished, I will start another new navigation plan.
I have also been adding the banner links, webs and History Channel
back. Do they actually help? They sure seem to slow down the opening
of the home page.
Subject: Re: Web site dropped by Google
From: kilroy-ga on 24 Feb 2003 10:55 PST
To sgtcory-ga
That was quick! While I was responding to ss64-ga your answer came
in.
1. I do know what robots.txt is. I had one there that ALLOWED
spiders but I removed it because, as I understand it, having NONE
allows spiders without reading it. I know it CAN be read by search
engines because it has good position in AltaVista, Ask Jeeves, DMOZ,
Looksmart, Overture, etc.
2. No, as you can see from the response to ss64-ga, I just put the
Site Map there. Hopefully it will be finished by tomorrow.
Subject: Re: Web site dropped by Google
From: ss64-ga on 27 Feb 2003 14:40 PST
OK firstly about 'not even the home page gets listed' - if (as is
likely) much of the home page is navigation rather than content then
this will probably cause a low page rank (a low enough rank => no
listing). Once sub pages are getting indexed then these add to the
ranking of the home page.
As for mozilla, I was finding that the links opened in pop-up windows,
where the text didnt quite fit in the size of window provided - by
scrolling around the small window you could follow links to other
pages - that then did an even worse job of fitting in the small window
(I gave up at that point)
Trying again today - I couldnt load the site at all (are you having
hosting problems or is it down for maintenance?)
Subject: Re: Web site dropped by Google
From: kilroy-ga on 27 Feb 2003 19:53 PST
To ss64-ga
First, thanks again for the tip about the Site Map. It was a job but
it is there now.
This is so frustrating! Everyone is hung up on ranking. I want to
solve the THERE problem - then I’ll start worrying about ranking
again! As to ranking, let’s look at a few comparisons to maybe put it
in perspective
If you search for Kilroy Was Here on Google, the top of the list is:
http://kilroy.blogspot.com/
It doesn’t show many total hits it has but my site has almost 400,000.
Google says 18 sites link to it, Lycos 23 and AltaVista Zero. Google
shows NO sites linking to mine but Lycos shows 227, AltaVista shows
251. Still less than mine on all counts. Maybe their links are
better quality of mine which would account for a higher ranking but
not the complete absence of mine.
If you search for kilroywashere on Google, the top of the list is:
www;kilroywashere.com. His counter shows that his total hits are
24064, mine has almost 400,000. Google shows Zero sites linking to
it. Lycos shows 188 and Alta Vista shows 145. Still less than mine
on all counts. Again, maybe their links are better quality of mine
which would account for a higher ranking but not the complete absence
of mine.
It seems apparent that Google has a problem seeing both Kilroywashere
and seeing links to it. That seems to be the problem not the usual
ranking problems.
As to the first page having no content, just navigation, look again.
It has a lot of content certainly a lot in comparison to the top
listed www.kilroywashere.com. It has NO content and almost no links.
Just 3 widely spaced lines that are simply links plus a counter.
I was unable to get on the site also for a while this afternoon. I
went to my host’s web site to find out what was wrong but it was down
also. They claim a 99.99+ up time. Maybe this was their .01
downtime. It came back shortly thereafter.
I will get a Mozilla browser and fix the problems. I check it with
IE6, Netscape 7, netscape 4.79 and WebTV. Now I will start with
Mozilla too. Did you try another browser? Did it work alright in it?
Subject: Re: Web site dropped by Google
From: kilroy-ga on 01 Apr 2003 15:26 PST
To sgtcory-ga
To ss64-ga
Well, we have passed another Google Dance and still not there. In
addition to your suggestions, Sarg, I have added another Links page
and added many more links to Kilroy including the Library of Congress,
Veterans History Project.
ss64, your suggestion about the Site Map was invaluable and, although
a big job, it will probably increase other search engine ranks (not
bad already.
As far as I'm concerned, both of you could claim the answer. Even
though the specific one wasn't answered, you helped a lot. I
understand we are not allowed to contact each other, so tell me how to
arrange that. I give up on getting the answer. My final effort was to
change nameservers. Who knows? That may help.
Subject: Re: Web site dropped by Google
From: sgtcory-ga on 01 Apr 2003 18:17 PST
This question already expired. I didn't offer an answer yet ;-) Most
of my comments were leading up to a fully qualified $50.00 answer. If
you would like to open this question again, and get your moneys worth,
I would be glad to complete an answer for you that will more than
likely help you in your efforts.
Thanks again!
SgtCory |
