On Fri, Nov 07, 2008 at 03:42:49PM -0500, Ofer Inbar wrote:
> We track the number of messages in each postfix queue on our
> mailservers using a program I've written. For most queues,
> it simply does a readdir() and counts all files whose names don't
> begin with ".", which is quick and efficient. For the deferred queue,
> it does a find-style walk through the directory tree. Unfortunately,
> on our fallback relays the deferred queues can get very large, and
> walking the tree becomes a long and expensive endeavor.
>
> By far the largest amount of time and I/O is spent calling stat() on
> hundreds of thousands of queue files to determing that they're simple
> files rather than directories to descend into. I could avoid that two
> ways:
When a directory's link count is 2 it has no sub-directories, and you
don't have to lstat() the files. This is how the find(1) leaf node
optimization works on Linux (you can suppress it with -noleaf).
When a directory's link count is > 2, it has "n-2" sub-directories,
and you can stop calling lstat() after finding that many sub-directories.
> 1. If I could tell simply from the filename whether it's a queue file
> or a subdirectory: for example, if the name is 3 characters or
> longer it's a queue file, shorter than 3 it's a directory.
>
> 2. If the tree structure were static enough that I could make
> assumptions about it: for example, every file in the top level is a
> subdirectory, every file in a subdirectory is a queue file.
As Wietse said, with the current Postfix, hash directories have
single-letter names, so you can also:
cd /var/spool/postfix; find active ! -name '?' | wc -l
this gives a number one higher than the true result. With custom code,
You can do a bit better than "find", by not calling stat on the hash
directories either, if you see a single letter (hex-digit 0-9A-F)
filename, assume it is a directory). Since you'll then be using
the directory anyway, the lstat() call has no real penalty, just
brings the inode in core a bit earlier. So in practice, find(1)
is good enough (if it has "leaf" optimization support, as with
say gfind on systems with older native find(1) versions).
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[EMAIL PROTECTED]>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
