I asked a question (http://answers.google.com/answers/threadview?id=402277)
a couple of weeks ago about a script to sort through some .csv files.
I got a great script which works perfectly for me on OS X. Until
recently I haven't needed to run it on Windows, so at the time didn't
bother testing it on that platform.
Anyway, now that I have tried it on XP, i'm getting an error message.
I'll quote the output I get when I run the script:
starting grok.py
enter a file name, or nothing to quit > [file name went here]
reading from file "c:\[file]"
discarding header
read 1071 lines
making result folder "c:\results.2004.09.29"
making result file "c:\results.2004.09.29\[filename].csv"
debug: (<class exceptions.IOError at 0x008646F0>, <exceptions.IOError
instance at 0x00A7ECD8>, <traceback object at 0x00A7ED00>)
error: failed to write line 1071 of
"c:\results.2004.09.29\[filename].csv"
enter a file name, or nothing to quit >
i've tried switching the read/write method from binary to text, as
recommended in the answer above, but it didn't work. If anyone could
help me out on this it'd be appreciated. |
Request for Question Clarification by
leapinglizard-ga
on
30 Sep 2004 14:20 PDT
That's a peculiar error, coming in the middle of a file like that. One
expects errors to occur at the very beginning or at the very end of a
file.
What happens if you pass grok.py the same file under OS X?
Do you have enough disk space on the Windows box?
Also, please add the following lines to the except clause where it
says, 'error: failed to write [etc.]'.
print 'attempted to write: '+(row+[CUSTOM_FIELD_VALUE])
I'd like to help you diagnose and correct this problem.
leapinglizard
|
Request for Question Clarification by
leapinglizard-ga
on
30 Sep 2004 14:22 PDT
Wait a second. The error isn't coming in the middle of the file at
all, but at the very end. I suspect there's something wrong with the
last line of the .csv file. Is there any way for you to post a weblink
to the file, if only temporarily?
leapinglizard
|
Request for Question Clarification by
leapinglizard-ga
on
30 Sep 2004 14:30 PDT
It's not necessarily the last line of the .csv input file that's
faulty, of course, but the last row that's meant to go into the .csv
output. Anyway, the key is to find out what it is about that last row
that makes the script choke. It shouldn't be very difficult to figure
out if you print the row in question so that we can take a good look
at it.
In fact, it would be even more helpful if you temporarily changed the
inner for loop in the try clause from
for row in h[key]:
line_num += 1
writer.writerow(row + [CUSTOM_FIELD_VALUE])
to the following.
for row in h[key]:
line_num += 1
print 'line_num = %d, row = %s' % (line_num, row)
writer.writerow(row + [CUSTOM_FIELD_VALUE])
When you get the error, copy the last five or ten lines of 'line_num =
[...]' output and post them here so I can take a look.
leapinglizard
|
Clarification of Question by
superunknown-ga
on
30 Sep 2004 15:48 PDT
OK I'll go through this tommorrow and see what happens. thanks leapinglizard.
|
Clarification of Question by
superunknown-ga
on
01 Oct 2004 11:29 PDT
OK here is the output I get when running the revised script on XP:
reading from file "c:\csv\testcsv.csv"
discarding header [HEADER]
read 1306 lines
making result folder "c:\csv\results.2004.10.01"
making result file "c:\csv\results.2004.10.01\result_[filename went here].csv"
debug: (<class exceptions.IOError at 0x00864750>, <exceptions.IOError
instance at 0x00D3E8C8>, <traceback object at 0x00D3E8A0>)
error: failed to write line 1306 of
"c:\csv\results.2004.10.01[filename went here].csv"
Traceback (most recent call last):
File "C:\csv\grok.py", line 168, in ?
print 'attempted to write: '+(row+[CUSTOM_FIELD_VALUE])
NameError: name 'row' is not defined
Looking at the last two lines I'm guessing that doesn't give you the
information you need. If you want I could put up a temporary link to
the test csv file I'm using so you could look at that. The same
modified script, with the same input file, works perfectly on OS X.
I'm definetely not out of disk space on the windows box.
|
Request for Question Clarification by
leapinglizard-ga
on
01 Oct 2004 11:47 PDT
Yes, please link to the test file. I'll let you know once I've downloaded it.
leapinglizard
|
Clarification of Question by
superunknown-ga
on
01 Oct 2004 13:01 PDT
Great, here's the link:
http://81.6.227.198/~nick/testcsv.csv
|
Request for Question Clarification by
leapinglizard-ga
on
01 Oct 2004 13:10 PDT
I've downloaded the file. I'll analyze it shortly.
Just to make sure we have exactly the same file, can you confirm the
character count or the md5 checksum on your Windows box?
> wc testcsv.csv
1306 16768 186593 testcsv.csv
> md5sum testcsv.csv
5dcd373b7bb651674722c4c277964aa0 testcsv.csv
leapinglizard
|
Clarification of Question by
superunknown-ga
on
01 Oct 2004 13:19 PDT
I don't have the windows machine to hand, but checking the file on my
mac, I'm getting a different wordcount and md5sum:
[~] wc testcsv.csv
1305 16768 186591 testcsv.csv
[~] md5sum testcsv.csv
3acc64f99e10f787c8e17cb443a50e99 testcsv.csv
Weird.
|
Request for Question Clarification by
leapinglizard-ga
on
01 Oct 2004 13:30 PDT
Are you quite sure the file you posted is the same one that causes
grok.py to crash? The one I downloaded from you contains 1306 CSV
rows, while the one that causes the crash has 1071. Furthermore, the
one I have doesn't produce anywhere near 1071 lines for a single
output file, whereas the error seems to have occurred right on the
1071th line in your case.
leapinglizard
|
Clarification of Question by
superunknown-ga
on
01 Oct 2004 14:23 PDT
A different file was used to generate the first output I quoted. The
second output used the same file as you have as its input. Both files
caused the same error to occur under windows, but both work fine under
OS X. Sorry to confuse you. As you can see from the test csv, no
output file should ever contain more than about forty rows.
Hope this helps.
|
