For a research project on the Internet search market, I would like you
to look at promising new technology in Internet Search.  What
technology is likely to be the next Google -- neural network search?
Personalised search? Visual search (like Kartoo).  What else?

You may wish to start with the search engines listings directory of
Yahoo and Dmoz and look at the listed companies for innovative search
technology.

I am not interested in the marketing and monetisation side of the
search market -- only in technology which will enable users of the
future to find stuff they want, faster and better than currently.

Do not cover any of these search engine companies: Google, FAST,
AskJeeves/Teoma, Altavista, Inktomi.  They are way too mainstream. 
What I want is for you to find a small startup, probably in seed
funding, probably staffed by a bunch of PhDs, with big ideas.

What I want from you is a list of innovative companies, ranked by what
you think of the quality of their technology, with a paragraph about
why they are special.

Please also include a list of all companies surveyed.

Be careful not to include any search company which does not own its
own engine (eg metacrawler, webcrawer, looksmart which is powered by
Inktomi)

---

Request for Question Clarification by pelican-ga on 10 Jan 2003 12:05 PST

Hello victorchua-ga:

This is a fascinating question.  I have a good answer for the basic
question (your first paragraph), but the additional information you
request in subsequent paragraphs would take (in my estimation) time
and effort far exceeding $50.  I could answer the basic question for
$50; but I could not collect, analyze, and document the entire
waterfront for less less than $200.

Sincerely,
pelican-ga

---

Clarification of Question by victorchua-ga on 12 Jan 2003 04:16 PST

Please go ahead and answer the first paragraph for $50.  

We did not expect a very lengthy answer for the second part of the
question - I was thinking three to four companies, one side of a sheet
of paper in total. However, I agree it's quite a lot of work because
you have to go through so many companies.  If we still require this
part of the question answered after you have answered the first part,
we will happily pay the $200.

I agree it's a fascinating question!

---

Request for Question Clarification by pelican-ga on 12 Jan 2003 17:50 PST

OK -- this is a good opportunity for me to learn together with you, so
please allow me a couple of days to fully validate the answer I have
in mind.  Any comments or suggestions from fellow researchers would be
most appreciated.

Kind regards,
pelican-ga

---

Clarification of Question by victorchua-ga on 13 Jan 2003 01:14 PST

Go ahead, and I look forward to your response in a few days.
I'm sure that the Google community will be interested in the answer to
this question too!  :-)

---

Request for Question Clarification by pelican-ga on 13 Jan 2003 07:09 PST

:-) I know ... hope this will not be my last answer!

pelican-ga

Hello:

After some extensive research I have been able to track down some of
the groups or companies that are close to bringing the revolution in
the search technology. The three emerging groups are listed along with
a summary of why are they above the competition.

DARPA http://www.darpa.mil

Project Genoa: 

The DARPA Information Awareness Office (IAO) is developing technology
that will promote sharing, collaborating, and reasoning to convert
nebulous data to knowledge and actionable options. The highlight of
the project is a “thematic” search engine called Athens that
complements traditional search engines by allowing users to find
nuggets of information in large collections of documents without
having to construct a complicated query, and breaks up result pages
into chunks of relevant information.

. A thematic search engine is more efficient for two reasons. First,
it allows the user to specify keywords one at a time and exposes the
search index by providing all related keywords and the amount of
information that would be returned at each step. Users can build
search queries incrementally, selecting additional search terms from a
list – there is less information to sift through to find the
information one needs, and one never gets 10,000 hits on a query,
which is the frequent result of using ordinary search engines. Second,
a thematic search engine also reduces the information returned by
breaking up HTML pages into smaller units, e.g., paragraphs or a few
sentences. With a standard search engine, the smallest unit of
information is a complete page, even though, most of the time, an
analyst’s question is very specific. With the standard engine, the
analyst has to scan a lot of irrelevant information to find the
desired bit.

Along with the Defense Intelligence Agency, various other defense
organizations are actively sponsoring the development of this new
technology.

A detailed list of the project can be found at
http://calder.ncsa.uiuc.edu/ACCESS/PPT/001213mscmc/Project_Genoa_White_Paper.doc

A prototype system demonstration of early components is available on
request at the DARPA Technology Integration Center.

STREAMLOGIC http://www.streamlogic.com

Streamlogic's Feed Management Technology

According to the vice president for business development for
streamlogic Inc., "Instead of archiving data and running search
queries through it, we archive search queries and run data through it.
It's a search engine on its head.”
The advantage of an inverted search engine, he claims, is that it's
6,000 times more efficient than the conventional approach. It can
handle huge volumes of data that would be expensive or impossible to
process using the standard method of loading data into an archive,
indexing it and then retroactively querying it.
Los Altos Hills, Calif.-based Streamlogic's feed-monitoring technology
"strains" the information through query rules in real time,
eliminating the archival requirement entirely. A demonstration at
www.streamlogic.com runs all the postings to some 50,000 Usenet news
groups—10 postings per second, or 2GB per day—through a database of
user-specified topics and instantly sends an alert every time one of
those topics appears in a post. It also turns unstructured information
into data that can put into a relational database for further
analysis.
A feed-processing engine plucks out information based on
user-specified topics or keywords. A feed analysis engine uses
statistical techniques to analyze, categorize and summarize
information for identifying trends, advertisement-targeting and other
applications. The engine improves with use as it learns the most
relevant words and phrases, says Streamlogic.

For more information, please visit
http://www.streamlogic.com/solutions/

FAST http://www.fastsearch.com
Search Engine: http://www.alltheweb.com

Why the top companies of the United States use Fast?
http://www.searchengineshowdown.com/features/fast/
BECAUSE It might be the next google. The engineers at this search
engine project have already claimed one of the best search rankings
award behind google.
http://www.fastsearch.com/press/press_display.asp?pr_rel=26
But there is more to it. AlltheWeb is the technology demo and Research
& Design sandbox for testing new search features
http://www.alltheweb.com. It includes enhanced features such as, more
than 2.1 billion web pages, 118 million multimedia files, 132 million
FTP files, two million MP3s, 15 million PDF files and supports 49
languages, making it one of the largest search engines available to
search enthusiasts.

Fast is implementing major features VERY FAST into its system. A
summary is provided at
http://searchenginewatch.com/searchday/01/sd1113-fast.html


Here is an article which shows the inner workings of this search
engine.
http://searchenginewatch.com/searchday/02/sd1031-in-fast.html

Because of the intense development in this search engine technology,
Fast is the number one choice for the 21st century China. Hers is an
article WHY?

http://www.fastsearch.com/press/press_display.asp?pr_rel=26

An interview with one of the top executives is located at:

http://www.europemedia.net/shownews.asp?ArticleID=8123
The best of search engine technology:

http://wi-consortium.org/

Useful Articles:

http://www.wired.com/news/business/0,1367,36574,00.html

http://www.computerworld.com/databasetopics/data/story/0,10801,70041,00.html

Following websites helped me to get major data:

http://www.searchenginewatch.com
http://www.


Hope you will find the answer useful. It is a very interesting topic.
Please clarify if you need more information. I am willing to work with
you. Thanks for asking.

Sincerely,
leader-ga.

---

Request for Answer Clarification by victorchua-ga on 14 Jan 2003 09:09 PST

Thanks for your work, which we think is first-rate.  At the moment we
don't need the second part of the question (a trawl through lots of
search engine firms) answered, but we'll let you know if things change
and we do.

---

Clarification of Answer by leader-ga on 14 Jan 2003 09:39 PST

Hello:

Thanks for your kind words. If needed, I will put every effort to
satisfy your requirements in the future.