On 10/31/2009 7:39 PM, Stan Hoeppner wrote:
Noel Jones put forth on 10/31/2009 1:12 AM:
Each lookup table requires overhead. 30 separate tables requires
considerably more overhead than one table. The size of the dataset
doesn't change, it's the overhead that gets smaller. The more
concurrent smtpd processes running, the more it matters.
I guess I was kinda looking for the technical definition of 'overhead'
here. If the overhead per file is a straight percentage, then it
shouldn't matter if it's one big file or 30 smaller files.
"overhead" is pretty much independent of file size. Each
extra lookup table is another open file, buffer space, another
library call (that might allocate its own buffer), and an
additional lookup. In a tiny environment it's not enough to
make a difference -- probably not enough to even notice.
But one lookup always beats two lookups, one open file always
uses less resources than two open files. Run enough smtpd
processes and it starts to make a difference; at some point it
can make a critical difference. And it's always "prettier"
to do one thing one time vs. doing the same thing over and over.
What is the
actual memory footprint overhead that's being added by Posfix for each
map file?
Measure it on your system. Probably not a lot, but it adds up.
And is the amount of overhead the same for all map types, or
different/variable depending on map type?
Yes, different table types use different system libraries and
have different resource requirements. But for each table type
it's a fairly constant value unaffected by table size (very
small tables might use a little less RAM under some conditions
-- but this really doesn't matter).
Is it something like 3 bytes
of overhead added for every 10 bytes in a map file, for example?
No, it's fixed.
I can
understand that having everything in one map file, or few map files,
compared to many, can increase lookup performance due to indexing
efficiency. What I don't understand is how having fewer larger map
files decreases memory footprint.
Fewer files = less overhead. Size doesn't matter here.
Also, given that cidr files aren't postmap'd, is it safe to assume they
aren't indexed? If they aren't indexed, how does putting the contents
in a single file speed up searching?
One lookup beats two every time.
If they are indexed when Postfix
starts or reloads, then why wouldn't Postfix created a single index
_after_ reading in all the data from all the cidr files, creating one
internal program table?
How is postfix supposed to know you want all these tables
treated as one? Anyway, postfix would still need to open the
files etc. so all you would save is the extra lookups. It's
easy enough for you to control by building the tables the way
you want them used that postfix shouldn't be guessing.
-- Noel Jones
