I am using Redhat Fedora Core 2 and I am implementing a file storage
system. I will give a unique ID to each file, hash the ID and store it
to the respective directory. so if there are 10,000,000 files, and
SUPPOSE the optimal maximum number of files that a directory can hold
is 1000, i will be storing the 42455th files at /0/42/42455.ext.

So my question is, what is the optimal maximum number of files that a
directory can hold in fedora core 2 ext 3 file system? I need that
number so that the least number of levels of folders is needed. Please
post the answer here with supportive information.

---

Request for Question Clarification by efn-ga on 19 Sep 2004 22:15 PDT

Hi fokfok,

I think I have found the maximum number, but I don't understand the
"optimal" part of the question.  Could you explain what you mean by
"the optimal maximum"?  Do you believe there is some range of file
counts that is optimal for something and you want the largest number
in that range?  If so, for what is the range optimal?

--efn

---

Clarification of Question by fokfok-ga on 19 Sep 2004 23:44 PDT

well, there is no maximum number for a directory right? I believe one
can upload unlimited files to a directory under unix. so my question
is, what is the maximum file number of a folder, so the number of disk
read will be minimum when the system is reading down the directory
tree structer for a specific file? (/0/42/42455.ext)

Or using my limited computer operating system knowledge, the question
can be rephrase as... What is the maximum number of files a single
directory index file can hold? (so that any file's position in a
directory can be found by just one read).

OK, I think I understand the question better now, but I find some of
your assumptions questionable.

I think there is a maximum number of files per directory, at least in
the ext3 file system.  If only that were your question, I could sell
you an answer!

I doubt that the data for a directory is stored in a number of "index
files," any one of which, and no more, can be read in one disk
operation.  Actually, a directory is a file, and in ext3, a directory
is a linked list of variable-length entries, so the length of the file
names you use is a factor in the analysis.  You might be able to
improve performance by leaving the ".ext" off the file names.

In current systems, the operating system or the file system may cache
data in main memory and the disk drive hardware may have a cache of
its own, so the number of relatively slow electromechanical disk
operations required can be hard to predict.  It might be more
effective to test various directory size values than to rely on a
theoretically optimal number.

Still, it is understandable that you would be interested in the
optimal number.  Unfortunately, I don't have the time and file system
wizardry to figure it out myself.  I hope someone else here can help
you.  If not, you might try asking in a Linux forum.

Good luck,

--efn

Yes, there are maximum numbers of files in directories. Unfortunately,
I am not familiar with Linux. But on BSD you usually type tunefs to
change various tunable filesystem settings such as average filesize,
whether the operating system is tuned for space allocation or time
etc. This feature should also be available for ext2/ext3 file systems
also. You can try to see if tunefs exists for Linux also, or you can
try a search in google to see if you can find the equivalent tool for
Linux. Hopefully this can help.