Hi, I had an unexpected surge of mail last night, and I see the following entries in my log (captured via mail.debug in syslog):
Jul 11 05:30:48 rust mail:info postfix/smtpd[409770]: connect from d03relay05.maildomain1.somewhere.com[a.b.c.107] Jul 11 05:30:48 rust mail:info postfix/smtpd[409770]: NOQUEUE: reject: MAIL from d03relay05.maildomain1.somewhere.com[a.b.c.107]: 452 4.3.1 Insufficient system storage; proto=ESMTP helo=<d03relay05.maildomain1.somewhere.com> Jul 11 05:30:48 rust mail:warn|warning postfix/smtpd[409770]: warning: not enough free space in mail queue: 4349952 bytes < 1.5*message size limit Jul 11 05:30:48 rust mail:info postfix/cleanup[1151160]: A1BB65415: message-id=<20090711103048.a1bb65...@rust.mydomain.somewhere.com> Jul 11 05:30:48 rust mail:info postfix/smtpd[409770]: disconnect from d03relay05.maildomain1.somewhere.com[a.b.c.107] Jul 11 05:30:48 rust mail:info postfix/qmgr[741482]: A1BB65415: from=<double-bou...@rust.mydomain.somewhere.com>, size=833, nrcpt=1 (queue active) Jul 11 05:30:48 rust mail:info postfix/cleanup[1151160]: A41C05438: message-id=<20090711103048.a1bb65...@rust.mydomain.somewhere.com> Jul 11 05:30:48 rust mail:info postfix/local[1048674]: A1BB65415: to=<postmas...@rust.mydomain.somewhere.com>, orig_to=<postmaster>, relay=local, delay=0.02, relays=0.01/0/0/0.01, dsn=2.0.0, status=sent (forwarded as A41C05438) Jul 11 05:30:48 rust mail:info postfix/qmgr[741482]: A41C05438: from=<double-bou...@rust.mydomain.somewhere.com>, size=977, nrcpt=1 (queue active) Jul 11 05:30:48 rust mail:info postfix/qmgr[741482]: A1BB65415: removed Jul 11 05:30:48 rust mail:info postfix/smtp[1007772]: A41C05438: to=<aixto...@us.somewhere.com>, orig_to=<postmaster>, relay=d03av02.maildomain1.somewhere.com[a.b.c.168]:25, delay=0.22, delays=0/0/0.13/0.09, dsn=2.0.0, status=sent (250 2.0.0 n6BAUm0r007497 Message accepted for delivery) Jul 11 05:30:48 rust mail:info postfix/qmgr[741482]: A41C05438: removed Jul 11 05:30:49 rust mail:info postfix/smtpd[1056984]: connect from d01relay03.maildomain2.somewhere.com[a.d.e.235] Jul 11 05:30:50 rust mail:info postfix/smtpd[1056984]: NOQUEUE: reject: MAIL from d01relay03.maildomain2.somewhere.com[a.d.e.235]: 452 4.3.1 Insufficient system storage; proto=ESMTP helo=<d01relay03.maildomain2.somewhere.com> Jul 11 05:30:50 rust mail:warn|warning postfix/smtpd[1056984]: warning: not enough free space in mail queue: 4349952 bytes < 1.5*message size limit Jul 11 05:30:50 rust mail:info postfix/cleanup[1151160]: 27B265415: message-id=<20090711103050.27b265...@rust.mydomain.somewhere.com> Jul 11 05:30:50 rust mail:info postfix/smtpd[1056984]: disconnect from d01relay03.maildomain2.somewhere.com[a.d.e.235] Jul 11 05:30:50 rust mail:info postfix/qmgr[741482]: 27B265415: from=<double-bou...@rust.mydomain.somewhere.com>, size=825, nrcpt=1 (queue active) Jul 11 05:30:50 rust mail:info postfix/cleanup[1151160]: 29DFB5438: message-id=<20090711103050.27b265...@rust.mydomain.somewhere.com> Jul 11 05:30:50 rust mail:info postfix/local[942242]: 27B265415: to=<postmas...@rust.mydomain.somewhere.com>, orig_to=<postmaster>, relay=local, delay=0.01, delays=0.01/0/0/0.01, dsn=2.0.0, status=sent (forwarded as 29DFB5438) Jul 11 05:30:50 rust mail:info postfix/qmgr[741482]: 29DFB5438: from=<double-bou...@rust.mydomain.somewhere.com>, size=969, nrcpt=1 (queue active) Jul 11 05:30:50 rust mail:info postfix/qmgr[741482]: 27B265415: removed Jul 11 05:30:50 rust mail:info postfix/smtp[401414]: 29DFB5438: to=<aixto...@us.somewhere.com>, orig_to=<postmaster>, relay=d03av01.maildomain1.somewhere.com[a.b.c.167]:25, delay=0.2, delays=0/0/0.12/0.08, dsn=2.0.0, status=sent (250 2.0.0 n6BAUotL024396 Message accepted for delivery) Jul 11 05:30:50 rust mail:info postfix/qmgr[741482]: 29DFB5438: removed Jul 11 05:30:51 rust mail:info postfix/smtpd[585982]: connect from d01relay03.maildomain2.somewhere.com[a.d.e.235] Jul 11 05:30:51 rust mail:info postfix/smtpd[585982]: NOQUEUE: reject: MAIL from d01relay03.maildomain2.somewhere.com[a.d.e.235]: 452 4.3.1 Insufficient system storage; proto=ESMTP helo=<d01relay03.maildomain2.somewhere.com> Jul 11 05:30:51 rust mail:warn|warning postfix/smtpd[585982]: warning: not enough free space in mail queue: 4345856 bytes < 1.5*message size limit Jul 11 05:30:51 rust mail:info postfix/cleanup[1151160]: CB27D5415: message-id=<20090711103051.cb27d5...@rust.mydomain.somewhere.com> Jul 11 05:30:51 rust mail:info postfix/smtpd[585982]: disconnect from d01relay03.maildomain2.somewhere.com[a.d.e.235] Jul 11 05:30:51 rust mail:info postfix/qmgr[741482]: CB27D5415: from=<double-bou...@rust.mydomain.somewhere.com>, size=828, nrcpt=1 (queue active) Jul 11 05:30:51 rust mail:info postfix/cleanup[1151160]: CD5BF5438: message-id=<20090711103051.cb27d5...@rust.mydomain.somewhere.com> Jul 11 05:30:51 rust mail:info postfix/local[1048674]: CB27D5415: to=<postmas...@rust.mydomain.somewhere.com>, orig_to=<postmaster>, relay=local, delay=0.02, relays=0.01/0/0/0.01, dsn=2.0.0, status=sent (forwarded as CD5BF5438) Jul 11 05:30:51 rust mail:info postfix/qmgr[741482]: CD5BF5438: from=<double-bou...@rust.mydomain.somewhere.com>, size=972, nrcpt=1 (queue active) Jul 11 05:30:51 rust mail:info postfix/qmgr[741482]: CB27D5415: removed Jul 11 05:30:52 rust mail:info postfix/smtp[938054]: CD5BF5438: to=<aixto...@us.somewhere.com>, orig_to=<postmaster>, relay=d01av02.maildomain2.somewhere.com[a.d.f.216]:25, delay=0.33, delays=0/0/0.19/0.13, dsn=2.0.0, status=sent (250 2.0.0 n6BASKnT013971 Message accepted for delivery) Jul 11 05:30:52 rust mail:info postfix/qmgr[741482]: CD5BF5438: removed Jul 11 05:30:52 rust mail:info postfix/smtpd[438490]: connect from d03relay05.maildomain1.somewhere.com[a.b.c.107] Jul 11 05:30:52 rust mail:info postfix/smtpd[438490]: NOQUEUE: reject: MAIL from d03relay05.maildomain1.somewhere.com[a.b.c.107]: 452 4.3.1 Insufficient system storage; proto=ESMTP helo=<d03relay05.maildomain1.somewhere.com> Jul 11 05:30:52 rust mail:warn|warning postfix/smtpd[438490]: warning: not enough free space in mail queue: 0 bytes < 1.5*message size limit Jul 11 05:30:53 rust mail:info postfix/cleanup[1151160]: 2BB515415: message-id=<20090711103053.2bb515...@rust.mydomain.somewhere.com> Jul 11 05:30:53 rust mail:warn|warning postfix/cleanup[1151160]: warning: 2BB515415: write queue file: No space left on device [ip addresses and hostnames have been obfuscated since I am paranoid] I'm interpreting the "warning: not enough free space in mail queue: x bytes" message as postfix telling me that there are x bytes free on the filesystem. I can see that this number decreasing between 2AM (approx 15MB free) down to the above entry at 05:30:51 (approx 4MB free). The decrements happen in chunks of approx 4KB - and that matches with the block size of the filessystem (this is on AIX - JFS2 filesystem). However, between 05:30:51 and 05:30:52, the remaining 4MB of space disappears. There is only one email handled(according to the syslog) in that interval, and since that bounce message arrived in the postmas...@rust.mydomain.somewhere.com inbox, and it wasn't 4MB large (approx 2KB), I'm left scratching my head as to what happened within that second. Does anyone have any thoughts on what might have happened? Any pointers on what to look for will be appreciated. Thanks, Kenneth
