Is there anywhere I can obtain a word list of the english language
ordered by frequency of occurance in communication (i.e. the, and
would be near the top and rarely used words would be near the bottom).

---

Clarification of Question by challenger70-ga on 01 Mar 2005 19:15 PST

I need a list that has pretty much every word in the english language
(scientific and frequently used foreign words would be a plus, but not
necessary).

---

Clarification of Question by challenger70-ga on 01 Mar 2005 19:18 PST

Also, a frequency index such as a 1-20 scale or something is also acceptable

Challenger70-ga,

There is exactly such a list of the Top 200 words at this site:


http://www.edict.com.hk/lexiconindex/frequencylists/words2000.htm


based on an analysis of a corpus (a collection of texts) of more than
a million words.

As you suspected, "The" is at the top of the list, accounting for
6.88% of all the words in the corpus, followed by of, and, to, a...

"Down" is #100, "current" is #1000, and "generation" is #2000.

I trust this information fully answers your question.  However, please
don't rate this answer until you have everything you need.  If you
would like any additional information, just post a Request for
Clarification to let me know how I can assist you further, and I'm at
your service.

All the best,

pafalafa-ga


search strategy:  Used bookmarked sites for word frequencies

---

Clarification of Answer by pafalafa-ga on 01 Mar 2005 19:25 PST

Ooops.

I'm afraid I didn't see your qualifier -- about having EVERY word in
the English language -- until after I posted my answer.  Ships passing
in the night, and all that...

Have a look at the link I provided about the Top 2000 (not 200, as I
typo'd), and then please let me know if it comes close to meeting your
needs.

In the mean time, I'll be looking for more comprehensive resources,
and you can expect to hear back from me (probably tomorrow) with more
information.

Looking forward to hearing back from you...

paf

---

Clarification of Answer by pafalafa-ga on 01 Mar 2005 19:47 PST

By the way, you can get the word list for words 2000-5000 here:

http://www.edict.com.hk/lexiconindex/


3000 -- sympathy 

  
4000 -- terror 


5000 -- ladder

---

Request for Answer Clarification by challenger70-ga on 02 Mar 2005 22:10 PST

Yea, this is for a computer program so it really needs every word,
which would be anywhere from 75,000 to 150,000...Id say around 75,000
which is considered average vocab (yes, the average person does know
that many words) would be great.

---

Clarification of Answer by pafalafa-ga on 03 Mar 2005 04:44 PST

No wonder they call you Challenger!

Try this one:


ftp://ftp.itri.bton.ac.uk/bnc/all.num.o5



This is a list from the British National Corpus of all words -- in
frequency order -- that appear more than five times in the corpus.  I
estimate there are about 60-70,000 entries on this list.

A full description of this file can be found here:

http://www.itri.brighton.ac.uk/~Adam.Kilgarriff/bnc-readme.html


Note that a much longer list is available that includes all words in
the corpus, even at a frequncy of five or less appearances.  You can
download that file by clicing on the link that says:


For all.num.gz click here


I hope these are exactly the sort of thing you're after, but if you
have need for anything else, just let me know, and I'm at your
service.


paf

Hmm, this is why I'm always reading Google answers.  You run across
some interesting stuff.

A question like this really has no answer because it all depends on
the corpus used. Even if you somehow combine different corpora, you
really won't get something representative of all the ways written
English is used:

http://www.askoxford.com/asktheexperts/faq/aboutwords/commonest

Is the word corpus on the list?  :)

Please see the Wordcount website

http://wordcount.org/main.php

It contains an ordered list of the 86,800 most common words based on
the 100 million word British National Corpus. It also is an example of
artistic, yet functional site design.

I am surprised the researcher did not find this award winning site.

Balloonguy, unfortunately thats in shockwave format, and I would need
something that I can parse...im gonna poke around the site though and
see if I can get a downloadable list.