On 3 sept. 2010, at 13:02, Mark Martinec wrote: > Versions before amavisd-new 2.7.0 and SA older than 3.3.0 are > not particularly suitable for a pre-queue filtering setup. > The combined new features of 2.7.0, SA 3.3.* and the postfix > 'speed_adjust' made such a setup much better behaved. > > Please read the introductory sections of 2.7.0 (pre)release notes: > > http://www.ijs.si/software/amavisd/release-notes.txt
Very interesting in fact. Thank you.
What surprises me, is that I use an amavisd in BQCF on our inbound mail server
for years now, and it's working great. It's a dedicated server (XServe, Mac OS
X Server) with 2 dual core Xeons, and 2 SATA HD in RAID 1. It's extensively
protected via black/grey listing, but nevertheless I've absolutely no
performance problem with it.
It looks like my internal traffic is way more aggressive than the part of
inbound traffic that survives black/grey listing.
> >Sep 2 13:00:47 ru amavis[87682]: (87682-15) TIMING [total 257879 ms] -
>> SMTP greeting: 25055 (10%)10,
>
> 25 s to get a greeting probably means there were more incoming
> sessions than the number of available amavisd processes.
> The smtpd_proxy_options=speed_adjust can help here.
At this time, amavisd processes were more numerous than smtp's (70 smtp max, 80
amavisd children)
Will give smtpd_proxy_options=speed_adjust a try.
>> SMTP DATA: 24052 (9%)19,
>
> The 2.7.0-pre7 brings a 4-fold speedup in receiving of large mail
> messages over SMTP. On our mailer the receiving transfer rate
> (postfix->amavisd) is now about 32 MiB/s.
I hope it will be ported soon to freebsd.
>> check_init: 25053 (10%)29,
>
> Hard to say what is going on here. Either a very busy machine,
> or maybe access contention on a Berkeley DB.
>
>> gen_mail_id: 21050 (8%)37,
>
> A slow/overloaded SQL server?
no SQL.
>> mime_decode: 21063 (8%)45,
>
> A very busy machine, or huge mail - probably both.
Machine does not look busy at all during those problems. Load is under 0.5 and
CPU is 90% idle. Even small emails are affected.
>> AV-scan-1: 21058 (8%)53,
>
> Very busy machine or a very slow virus scanner.
>
>> decide_mail_destiny: 25156 (10%)63,
>
> Explicable only by a very much overloaded host, or memory shortage
> and swapping.
I see neither :(
>> fwd-connect: 64265 (25%)88,
>
> postfix too busy to accept another connection???
> Or perhaps you have too low a limit on the number of
> back-end smtpd processes.
I'll check this.
>> nop, it's mostly shared memory here, With 80 amavisd process, I've still
>> 2GB RAM free, and absolutely no swap. With 120 amavisd process and a max
>> of 120 smtpd, I still have more than 1.5 MB of free RAM, and no swap.
>
> If you are not running SpamAssassin, the 80 processes on a 4 Gb host
> is probably still reasonable. So it must be the CPU overload,
> not a memory exhaustion.
I made a blunder, the host has 6 GB RAM, not 4 GB.
At this very moment, I have switched back to previous settings:
- 50 amavisd children
- 50 max smtp
I leaves +3700 MB RAM free.
Amavisd behaves normally for the moment but it went crazy this morning again,
for few hours.
normal behavior:
Sep 3 14:39:20 ru amavis[21117]: (21117-08) TIMING [total 1635 ms] - SMTP
greeting: 2 (0%)0, SMTP EHLO: 1 (0%)0, SMTP pre-MAIL: 1 (0%)0, SMTP
pre-DATA-flush: 4 (0%)0, SMTP DATA: 2 (0%)1, check_init: 0 (0%)1, digest_hdr: 1
(0%)1, digest_body: 0 (0%)1, gen_mail_id: 1 (0%)1, mime_decode: 9 (1%)1,
get-file-type1: 16 (1%)2, parts_decode: 0 (0%)2, check_header: 1 (0%)2,
AV-scan-1: 8 (0%)3, spam-wb-list: 1 (0%)3, SA parse: 4 (0%)3, SA check: 1550
(95%)98, update_cache: 7 (0%)98, decide_mail_destiny: 1 (0%)98, fwd-connect: 5
(0%)99, fwd-xforward: 1 (0%)99, fwd-mail-pip: 3 (0%)99, fwd-rcpt-pip: 0 (0%)99,
fwd-data-chkpnt: 0 (0%)99, write-header: 1 (0%)99, fwd-data-contents: 0 (0%)99,
fwd-end-chkpnt: 2 (0%)99, prepare-dsn: 1 (0%)99, main_log_entry: 8 (1%)100,
update_snmp: 2 (0%)100, SMTP pre-response: 0 (0%)100, SMTP response: 0 (0%)100,
unlink-2-files: 1 (0%)100, rundown: 1 (0%)100
>>> Also, to RUN those 80 amavis threads, I would start with at least 4
>>> cores.
>>
>> I do agree about cores. The VM what tuned for 30-40 amavisd, I should
>> double the CPU, or remove amavisd and smtpd processes. I think I'm going
>> to remote amavisd/smtpd processes. I've found out few hours after sending
>> my question here that updating few perl modules could be the solution to
>> my problem. I'm writing "could be", because I'm still not sure it's THE
>> solution. After updating few perl modules (including NET::Socket) and
>> restarted amavisd, it immediately started to work great. It does not
>> guaranty it will not break under load again.
>
> Maybe, although I'm not aware of any performance-related problems
> with underlying perl modules. More processors would definitely help.
As said above, amavisd went very slow again today, so the module update didn't
help. I've added 2 CPUs, but it didn't help either. After that, the max
processing time for amavisd was around 130-150 seconds, instead of 250-260 seen
yesterday. But the proxy timeout warnings in the postfix logs were as numerous
as yesterday.
I've had to put every email of "mailgw" queues into hold for the symptoms to
disappear. "mailgw" is the postfix instance that rewrite recipient address for
emails that should be delivered in our domain. "mailgw" receives emails from
our inbound server (mx) and our smtp's ("smtp" and "smtp-liste").
In general, the traffic "mx"->"mailgw"->"smtp" or "smtp"->"mailgw"->"smtp" or
"smtp-liste"->"mailgw"->"smtp" (recipient address in our domain, but finally
rewritten into an external address) is very low.
These days, this traffic is higher, because addresses of +2000 users are
aliased to a remote domain (it's 7% of our active users, 3% of our total users
list), so we have many emails going from "mailgw" to "smtp". But I don't see
how few more emails can wreak havoc in amavisd.
That would be a good idea to plug "mailgw"-to-"smtp" traffic on an smtpd
without filtering.
Patrick PRONIEWSKI
--
Administrateur Système - SENTIER - Université Lumière Lyon 2
smime.p7s
Description: S/MIME cryptographic signature
