Hi,
Yes Google does limit the return, but it shows most of the sites
anyway. The limitation after so many 100's is on the domain and the
116,000 + is on URL. Remember that every page on your website is a
single URL, but your domain is only one. Since after 800 or so you
probably saw all the "domains" more than once the program stops there.
I ran a Perl program with your query "golf shops" using the Google
API, and it returned only 500. So this tells me there is a definite
limiting factor on how many results the engine is going to return to
you for a single query. I'll list the source for this here, just in
case you are interested.
#! /usr/local/perl -w
use WWW::Search;
my $key="INSERT YOUR GOOGLE KEY HERE";
my $search = WWW::Search->new('Google', key => $key);
$search->native_query("golf shop");
while (my $result = $search->next_result()) {
print $result->url, "\n";
#print $result->description, "\n";
}
The rest of the pages are available to you, what needs to happen
though is the query needs to be narrowed so that those pages become
more relevant and come closer to the the top. Doing this takes some
patience or a nice little Perl program that can weed through the ones
it already found, using different queries. It's very possible and kind
of fun if you are into this type of thing, or just really want to
know.
Thanks,
webadept-ga |
