I would like to know the number of web pages of content on the
Internet each year from 1995 through 2005. If that is not possible, a
substitute could be the amount of bytes of data on all the web pages
on the Internet over the past 10 years. It would also be helpful to
know the source of the data. |
Request for Question Clarification by
pafalafa-ga
on
07 Sep 2005 15:08 PDT
waiter-ga,
Fascinating question, but measuring the actual size of the internet
has proven to be a remarkably elusive exercise.
Search engines like Google routinely report the number of web pages
indexed (currently at about 8 billion for Google), but this represents
only a fraction of the online content, and the size of the fraction is
not really known.
The best and most reliable trend statistic I know of is the count of
domain hosts, which has risen from a few hundred in 1994, to almost
400 million at present:
http://www.isc.org/index.pl?/ops/ds/
Take a look at the graph (and some of the explanatory pages, like the
FAQs) and let me know if this meets your needs.
pafalafa-ga
|
Clarification of Question by
waiter-ga
on
07 Sep 2005 15:40 PDT
Thanks pafalafa-ga but the number of hosts is not exactly what I was
looking for. I am trying to get some estimate of the amount of
content on the web and how that has grown over time. Has anyone tried
to do this?
|
Request for Question Clarification by
pafalafa-ga
on
07 Sep 2005 16:29 PDT
The only estimate I'm aware of are those from the very well-known "How
Much Information" studies conducted at UC Berkeley. These estimated
the size of the internet for 2000 and 2003.
Here's the 2003 study:
http://www.sims.berkeley.edu/research/projects/how-much-info-2003/internet.htm
and here's some of the internet write-up:
===============
Published Estimates on Size and Character
Most size studies of the World Wide Web have focused on the number of
hosts connected to the network. Preliminary estimates of the amount of
data on the web have been made but have not been kept up to date or do
not have defined measurement methods. In our last survey we quoted a
page size estimate of 18.7 KB from a 1999 article in Nature Magazine
that was generated by statistically sampling web servers.
...We downloaded and analyzed the contents of 9,800 websites in order
to estimate the size of an average webpage and the contents of an
average website. These sites were chosen randomly from a list of 61
million URLS compiled by the Internet Archive...
...The sum of all the web site file sizes in our sample equals 33.1
GB. As this sample of 9,806 sites represents 0.02 percent of the 42.8
million web servers (according to the NetCraft Survey, as of August
2003), we may estimate the total size of the surface web as 167 TB
(95% confidence, +/- 1). Therefore the "deep Web" may be between
66,800 and 91,850 TB...
===============
It's not unreasonable to suppose that the size in terabytes has grown
at pretty much the same pace as the number of hosts. If that's the
case, then some size trends could be estimated from the host numbers.
Let me know if that's useful at all.
paf
|
Clarification of Question by
waiter-ga
on
07 Sep 2005 18:53 PDT
Paf, not the exact stats I needed but getting closer. I knocked the
price down to $10 if you want, I'd pay you that amount for the
information you provided. Thanks.
|
Request for Question Clarification by
pafalafa-ga
on
07 Sep 2005 19:26 PDT
Thanks for the offer...but I'm going to pass.
I'm never comfortable collecting a fee for providing information that
a client didn't really find met their needs.
You can leave this open (and even raise the price back up) to see if
anyone else can answer it for you. Or, you may prefer to cancel the
question.
All the best,
paf
|
