Google Answers: Web Crawling, Internet mining, Internet extraction tools

View Question

Q: Web Crawling, Internet mining, Internet extraction tools ( No Answer, 1 Comment )

Question

Subject: Web Crawling, Internet mining, Internet extraction tools
Category: Computers > Internet
Asked by: jonshurts-ga
List Price: $25.00

Posted: 05 Nov 2004 23:26 PST
Expires: 05 Dec 2004 23:26 PST
Question ID: 425165

Hi,

     I am looking for a web crawler/spider/bot that can search
websites that I specify. I am looking for a crawler that can be told
to startup on its own, check websites specified in a text file,
extract the data, write the data to a file and then terminate all on
its own. Please Help!!

Request for Question Clarification by tar_heel_v-ga on 12 Nov 2004 05:49 PST
When you say extract data, what do you mean?  Are you looking to
spider the entire site and write the urls to a text file?

Clarification of Question by jonshurts-ga on 17 Nov 2004 23:00 PST

Yes, I am looking to spider the entire site, and write any text,
including urls to a text file. Also, it would be great if the crawler
could get the names of the websites it needs to spider from a text
file created by the user.

Answer

There is no answer at this time.

Comments

Subject: Re: Web Crawling, Internet mining, Internet extraction tools
From: lordo-ga on 20 Nov 2004 22:42 PST

These are two good ones.
Win Web Crawler
http://www.tucows.com/preview/325173.html
Web Crawler
http://www.tomdownload.com/internet/download_managers/visual_web_task.htm

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.

Search Google Answers for

Google Home - Answers FAQ - Terms of Service - Privacy Policy