I need a list, or lists, of the most common words in English, listed
in order of popularity, i.e. starting with "the" and working down. The
lists should be as large as possible -- 20,000, 30,000 or even more,
if possible. The most important point is that I need to be able to
download the lists as text files. And ideally, I would like lists from
different domains, such as "Most common words in newspapers," or "Most
common words in academic
research." (This latter idea is optional, but would be nice.) Can
anyone send links of where to find these?

---

Request for Question Clarification by pafalafa-ga on 04 Aug 2006 17:50 PDT

boboleo-ga,

There are a lot of frequency lists out there (see the link in
comments, below), but there are suprisingly few easily-accessible
large lists.

Your best bet may be the wiktionary lists of words in Project Gutenberg:


http://en.wiktionary.org/wiki/Wiktionary:Frequency_lists


Click on the links where it says:


New list as of 4/16/2006: 
Wiktionary:Frequency lists/PG/2006/04/1-10000 
Wiktionary:Frequency lists/PG/2006/04/10001-20000 
Wiktionary:Frequency lists/PG/2006/04/20001-30000 
Wiktionary:Frequency lists/PG/2006/04/30001-40000 


There are also lists of a few thousand words based on the works of Shakepeare:


http://www.link75.org/mta3/curriculum/english/Shakes/index.html



Do these lists meet your needs?


pafalafa-ga

P.S.  The wiktionary link also has a list of 40,000 words from TV and
movie scripts, but they're available in discrete chunks of a few
thousand words at a time.

---

Clarification of Answer by pafalafa-ga on 04 Aug 2006 18:01 PDT

<<sigh>>

I meant my 'P.S' as a clarification, not as an Answer!


Let me know what you think of the links provided, and also, what
additional sorts of information you'd like, and I'll do my best.

However, I can always ask the editors to have my 'answer' removed, if
that is your preference...sorry for the mix-up,

paf

---

Clarification of Answer by pafalafa-ga on 04 Aug 2006 19:11 PDT

And here's another 5,000, by the by:


List of 2000 most frequently used words (Brown Corpus) 
http://www.edict.com.hk/lexiconindex/frequencylists/words2000.htm


List of the next 2000-5000 most frequently used words (Brown Corpus) 
http://www.edict.com.hk/lexiconindex/frequencylists/words2-5k.htm

Some of the material in this answer to a similar question may be of use to you:

http://www.answers.google.com/answers/threadview?id=470042