Hi there,

Last week you wrote a link parser to generate a list of URLs to
submit. The program is working fine but we would like a couple of new
features.

Are you ok for this?

---

Request for Question Clarification by joseleon-ga on 09 Dec 2003 08:24 PST

Hello, bmcompany:
  Sure, just tell me what do you need and you have it.

Regards.

---

Clarification of Question by bmcompany-ga on 09 Dec 2003 09:28 PST

Hello again!

at the moment, using the link parser, you choose a directory and it
reads the url from the top links and then copies that into a list,
appending all of the .htm file names to the URLS, until you have a
complete url (inc page name) ready for submission.

We now need to do the same but with several sites in one directory
(each site with around 50 pages. IE, one dir will have 400 pages, 8
clients (50 pages per client)

So, the program needs to look into each html file and extract the url
on an individual basis.

the list will look like this

http://www.domain1.com/page1.htm
http://www.domain1.com/page2.htm
http://www.domain1.com/page3.htm
http://www.domain1.com/page4.htm
http://www.domain2.com/page1.htm
http://www.domain2.com/page2.htm
http://www.domain2.com/page3.htm
http://www.domain2.com/page4.htm
http://www.domain2.com/page5.htm
http://www.domain3.com/page1.htm
http://www.domain3.com/page2.htm
http://www.domain3.com/page3.htm

Thats the first of 2 changes.

Secondly, for each client, one of the pages contains a list of links
to other pages (in a hidden layer). We need another field in the DB
which notes whether the page contains this hidden layer or not. If you
have a look at the files i sent you last time, you should be able to
find the hidden layer (as its the html file thats 1 or 2K larger than
the others).

If you could clarify the question to make sure i have made sense.

Thanks Again

bmcompany

---

Request for Question Clarification by joseleon-ga on 09 Dec 2003 09:50 PST

Hello, bmcompany:
  First of all, thanks for your confidence with me, I will try to make
it as fast and as better as possible, so here is what I understand do
you want me to do:
  
1. You want me to alter the previous procedure and use the domain name
stored "independently" on each file, that way, you can have in the
same directory many customer websites and generate reports for a
single customer or for several without need to change anything, right?

2. For the second change, I deleted your data files, so I would need
you to post them again, and you just want me to scan for that hidden
layer and store it's existance (or absence) in a field in the db. Do
you want this information on the text result also?

Regards.

---

Request for Question Clarification by joseleon-ga on 10 Dec 2003 00:48 PST

Hello, bmcompany:
  Should I start the modifications? Everything's ok?

Regards.

---

Clarification of Question by bmcompany-ga on 10 Dec 2003 01:01 PST

http://www.sesuk.net/ga/ga.zip

sorry for the delay, we're in the uk - so you're 5 hours or so behind.

The text output doesnt need the hidden layer info, just the db.

Perfect, please go ahead with the modifications. Please let me know
once you've downloaded the file and I'll remove them.

Thanks again

bmcompany

---

Request for Question Clarification by joseleon-ga on 10 Dec 2003 01:38 PST

Hello, bmcompany:
  I have downloaded the file, you can remove it safely, by the way,
I'm in Spain, so here is GMT+1 ;-)

I start the modifications right now.

Regards.

---

Clarification of Question by bmcompany-ga on 10 Dec 2003 02:22 PST

good stuff!

Hello, bmcompany:

You can download the improved version from this location:

http://www.qadram.com/Link_parser_improved.zip
  
Now you can mix all the websites you want on a single dir and results
will be extracted sorted and grouped by domain, also, in the database,
a new field has been created, called "layer", is boolean and stores
whether the file contains the hidden layer or not.

I hope this is what you need, but in any case, don't hesitate to
request for a clarification, I'm here to help you!

Regards.

---

Clarification of Answer by joseleon-ga on 10 Dec 2003 06:08 PST

Hello, bmcompany:
  Thanks for the tip!, I'm online most of the time, so you can post a
new question when you want.

Regards.

Perfect (worked for 30,000 pages no problem :)

next thing we want it to do is to have the URL and page name as 2
seperate fields. At that point the database needs to be split to 2
tables and normalised. Ill post that for you later on today if that
falls within your remit?