Google Answers Logo
View Question
 
Q: Perl dbmopen and dbmclose behavior ( No Answer,   2 Comments )
Question  
Subject: Perl dbmopen and dbmclose behavior
Category: Computers > Programming
Asked by: ebreak-ga
List Price: $15.00
Posted: 05 Apr 2004 17:17 PDT
Expires: 05 May 2004 17:17 PDT
Question ID: 325727
I've got two systems (1. RedHat 9.0 and Perl 5.8,,, 2. SunOS 5.8 and
Perl 5.0) using the same Perl program I wrote. I use the dbmxxxxx
functions to store various bits of information on about 12000+ text
files on each system.

On system 1 the data.db file is about 12 meg in size, on system 2 the
data.pag file is over 3 *gig* in size. Similiar info on both systems
and exactly the same program on both systems.

Why the *huge* difference?
Answer  
There is no answer at this time.

Comments  
Subject: Re: Perl dbmopen and dbmclose behavior
From: cfr-ga on 13 May 2004 03:22 PDT
 
DBM files are "files with holes". A file that contains "holes"
(sometimes called a "sparse" file) means that the application wrote
some data then seeked to another offset later in the file and wrote
some more data. On most (all?) Unix systems the part of the file that
was skipped will not be stored to disk and will be zero-filled if read
by another application.

I'd guess that the 3 GB file was either copied, archived and extracted
(tar), or transferred over the network (ftp/scp). See the --sparce
option of the GNU cp command for an example of how to efficiently copy
these files.

In this old Sun manpage (the first I found),

http://mirrors.ccs.neu.edu/cgi-bin/unixhelp/man-cgi?dbm+3

says,

---
     The .pag file will contain holes so that its  apparent  size
     may be larger than its actual content. Older versions of the
     UNIX operating system may create real file blocks for  these
     holes  when  touched. These files cannot be copied by normal
     means ( cp(1), cat(1), tar(1), ar(1)) without filling in the
     holes.
---

You'll also find that those files are platform dependent. The standard
dbm interfaces are really only for backward compatability now. You
should really consider moving to something like GDBM (which you can
bind to directly in Perl. See the AnyDBM module -- the good news is
that you'll probably only have to change a few lines of code to
switch)

http://aspn.activestate.com/ASPN/docs/ActivePerl/lib/AnyDBM_File.html

to get (among other things) longer key/value pairs, non-sparse and portable files.

http://www.delorie.com/gnu/docs/gdbm/gdbm_16.html

Good luck.
Subject: Re: Perl dbmopen and dbmclose behavior
From: ebreak-ga on 13 May 2004 04:32 PDT
 
The 3 GB file was created (on both systems) by a script I wrote.

I've tried to use other modules but I couldn't get any to work, the
program wouldn't compile. I don't remember the details now so I'll try
again.

Thanks for the comment cfr!

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy