On 09/04/2010 10:42 PM, Stan Hoeppner wrote:
Victor Duchovni put forth on 9/4/2010 7:33 AM:
What do you mean by "filters"?
Spam filters in the form of table lookups and dnsbl queries. I'm
currently processing
12,581 CIDRs
1,568 regular expressions (PCRE)
5 dnsbl lookups
per each inbound connection (assuming no hits). Obviously a hit in a
table early on in the processing order is going to cut the time down.
I'm just wondering how much time is being consumed by processing all of
these and the lookup delays to the dnsbl servers. Is it so small it's
irrelevant, or is it large enough it could become an issue as I add more
tables/entries and/or dnsbls, and inbound mail volume increases? Say I
add another 20k CIDR or 1,500 more PCRE entries?
That should be relatively easy to determine given a few messages of
which it is known where they hit the filters, and your normal delays values.
If the difference shows up in the pre-queue delay at all, you can
calculate stuff.
If it doesn't, it means the difference is < 1ms and I wouldn't worry
about it ;)
For best results, try a message that hits the first filter, something in
the middle, and something that doesn't hit at all.
DNSBL checks are by far the biggest impact, being orders of magnitude
slower than any other possible check - it has to do a DNS call and wait
for the result.
As for your original question, the combined processing time of all your
smtpd_* checks will still be reflected in the delay-"a" value (pre-queue).
Whatever time postfix itself adds for processing will be either static
or insignificant (unless you have lots of expensive maps) .
FYI this is an old dual CPU 500 MHz 32bit x86 machine. Now you may
understand my concern about running large numbers of PCREs and CIDRs per
connection. Even with these old CPUs, processing these tables may be a
piece of cake, I simply don't know. That's why I'm asking.
Mostly memory usage - all those are permanently loaded in memory.
CIDRs are nicely compacted, and should perform equivalent to a tree
search (matching closest-subnet-first).
Regexen cannot be optimized much, of course, so they will by their
nature be slower.
Why not run a few hundred messages in debug mode and record the
processing start-end times of all of those tests ?
It's not as if this will fluctuate over time.
Using any (or all) of the methods I mentioned should at least give you a
rough indication of the time taken for CIDR checks, Regex checks, and
DNSBL checks.
It may be that all of this is a total non issue, and if so, I'd like to
know that.
Thanks.
