Continuation of 323949

Let me know if you have any questions.

---

Request for Question Clarification by joseleon-ga on 13 Apr 2004 09:07 PDT

Hello, bmcompany:
  Just came back from the short easter holidays, I'm going to gather
all the requeriments and list them here before I continue.

Regards.

---

Request for Question Clarification by joseleon-ga on 13 Apr 2004 10:37 PDT

Hello, bmcompany:

These are the requeriments for the spider project:
-There will be a new option, called Spider a URL
-You will enter an URL of the site to spider
-The program will get that URL and will parse it for links, that will
become the 1st level
-Then, it will grab each URL recursively getting a list of links
inside those pages, that will become the 2nd level
-And it will repeat operation to get the 3rd level URLs
-The level will be configurable as a setup option
-When it finishes, it will show a form, filled with the gathered URLs,
to allow you submit them
-These urls will have a switch set to allow you differentiate them
-And there will be another new option, called Clients Own Pages, which
will be exactly as the Show URLs, but showing only clients pages

Please, tell me if this list is ok, and modify anything you need.

Regards.

---

Clarification of Question by bmcompany-ga on 14 Apr 2004 03:13 PDT

absolutly spot on (as usual)

Best of luck!

---

Request for Question Clarification by joseleon-ga on 15 Apr 2004 06:31 PDT

Hello, bmcompany:
  I have added the spider option to the interface, it spiders a site
with a 3 depth level and then show the results waiting for submission
to the database.

The depth level is not configurable yet and the urls are not submitted
to the database, I'm waiting for your review and I will finish it as
soon you give me the ok.

Regards.

---

Clarification of Question by bmcompany-ga on 16 Apr 2004 01:17 PDT

hi there,

few errors

this is spidering http://www.sesuk.net

Warning: file(http://www.winegiftcentre.net/Corkscrews.htm): failed to
open stream: HTTP request failed! HTTP/1.1 404 Not Found in
/home/bowmacsub/www/parse.php on line 454

Warning: join(): Argument to implode must be an array. in
/home/bowmacsub/www/parse.php on line 456

Warning: file(): php_network_getaddresses: getaddrinfo failed: Name or
service not known in /home/bowmacsub/www/parse.php on line 454

Warning: file(http://www.blackpool-self-catering.co.uk): failed to
open stream: No such file or directory in
/home/bowmacsub/www/parse.php on line 454

Warning: join(): Argument to implode must be an array. in
/home/bowmacsub/www/parse.php on line 456

Warning: file(http://www.keith-jones-ifa.co.uk/Independent_Financial_Advisers_Evesham.htm):
failed to open stream: Connection timed out in
/home/bowmacsub/www/parse.php on line 454

Warning: join(): Argument to implode must be an array. in
/home/bowmacsub/www/parse.php on line 456

---

Request for Question Clarification by joseleon-ga on 16 Apr 2004 01:20 PDT

Hello, bmcompany:
  Those errors are caused because there are links to pages that cannot
be opened, I mean, are dead links. I'm going to handle them correctly
and also to finish the feature list.

Regards.

Hello, bmcompany:

I have finished the spider option by:
-Time out errors are conveniently handled
-Customer urls are stored on the database
-There is a new option to show only the customer urls and the Show
Urls option doesn't show those urls
-There is a new setting on the setup option in which you can set the
depth level when spidering, be careful with this because a high depth
level can take a lot of time to complete

Please, let me know if everything works as you expected.

Regards.

perfect