Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-23 Thread Konrad Rzepecki
W dniu 22.01.2012 01:14, Stan Hoeppner pisze: On 1/21/2012 6:53 AM, Michael Tokarev wrote: On 21.01.2012 15:42, Michael Tokarev wrote: On 20.01.2012 16:01, Stan Hoeppner wrote: And having said all that, I think it'd be interesting to understand why the OP has this slow system. It should not be

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-21 Thread Stan Hoeppner
On 1/21/2012 6:53 AM, Michael Tokarev wrote: > On 21.01.2012 15:42, Michael Tokarev wrote: >> On 20.01.2012 16:01, Stan Hoeppner wrote: > [] >>> As it turns out the OP has Seagate LP drives which are not Advanced >>> Format 512/4096 drives. No alignment issue there. >> >> I never had/usd these so

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-21 Thread Michael Tokarev
On 21.01.2012 15:42, Michael Tokarev wrote: > On 20.01.2012 16:01, Stan Hoeppner wrote: [] >> As it turns out the OP has Seagate LP drives which are not Advanced >> Format 512/4096 drives. No alignment issue there. > > I never had/usd these so don't know. Apparently the LP drives > are more opti

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-21 Thread Michael Tokarev
On 20.01.2012 16:01, Stan Hoeppner wrote: > On 1/20/2012 1:50 AM, Michael Tokarev wrote: > >> Please excuse me for the somewhat harsh words, but except of the >> alignment issues which should be solved for once when partitioning >> and creating filesystem, the rest is a complete bullshit collected

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-20 Thread Stan Hoeppner
On 1/20/2012 1:50 AM, Michael Tokarev wrote: > Please excuse me for the somewhat harsh words, but except of the > alignment issues which should be solved for once when partitioning > and creating filesystem, the rest is a complete bullshit collected > from various forums where people does not unde

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-20 Thread Stan Hoeppner
On 1/20/2012 1:30 AM, Konrad Rzepecki wrote: > W dniu 20.01.2012 01:39, Stan Hoeppner pisze: >> On 1/19/2012 5:07 AM, Konrad Rzepecki wrote: >>> Yes, you have right. But I found recently, that disk mounted on my >>> server are slow 5.9K. My tests on in shows that they do fsync 1.5x-2x >>> slower th

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-19 Thread Michael Tokarev
On 20.01.2012 04:39, Stan Hoeppner wrote: [] >>> But that >>> alone isn't going to fix a 10x performance deficit. You've probably got >>> multiple factors degrading performance. >> >> Yes, you have right. But I found recently, that disk mounted on my >> server are slow 5.9K. My tests on in shows t

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-19 Thread Konrad Rzepecki
W dniu 20.01.2012 01:39, Stan Hoeppner pisze: On 1/19/2012 5:07 AM, Konrad Rzepecki wrote: Yes, you have right. But I found recently, that disk mounted on my server are slow 5.9K. My tests on in shows that they do fsync 1.5x-2x slower than 7.2K with quite often 5x-10x slower peak. Together with

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-19 Thread Stan Hoeppner
On 1/19/2012 5:07 AM, Konrad Rzepecki wrote: > W dniu 19.01.2012 08:15, Stan Hoeppner pisze: > >> To demonstrate that fsync alone shouldn't be a factor here, > > But it is. I've straced sendmail to "fsync" waiting lot of time. It was > 80% or more of queue time. > >> Queuing 15K messages took 6

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-19 Thread Konrad Rzepecki
W dniu 19.01.2012 08:15, Stan Hoeppner pisze: To demonstrate that fsync alone shouldn't be a factor here, But it is. I've straced sendmail to "fsync" waiting lot of time. It was 80% or more of queue time. Queuing 15K messages took 6 minutes 30 seconds on a single 7.2K drive, again while co

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Stan Hoeppner
On 1/18/2012 3:26 PM, Stan Hoeppner wrote: > On 1/18/2012 4:46 AM, Konrad Rzepecki wrote: >> I don't know why it take some much time, but it cannot be disabled >> (fsync) in postfix due to delivery guarantee. However, my mails (those >> ones) are not very important, and lost some of them is not a

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Andreas Berton
On Sat, 18 Feb 2012, Andreas Berton wrote: On Wed, 18 Jan 2012, Simone Ruffilli wrote: Il 18/01/2012 10:35, Ralf Hildebrandt ha scritto: > * Simone Ruffilli: > > >whenever I submit to my postfix server a mail having a massive > > >(~15k) > > >recipient list, it takes foreve

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Stan Hoeppner
On 1/18/2012 4:46 AM, Konrad Rzepecki wrote: > I have similar problem, and I suspect you may also hit it. In my case I > send ~10k-20k mails through sendmail wrapper. My problem is caused by > fsync done on EVERY queued mail. It take on my servers (4 core AMD @ > ext4 @ LVM @ RAID10 @ 8disk) ~ 0.2

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Wietse Venema
Konrad Rzepecki: > W dniu 18.01.2012 13:03, Wietse Venema pisze: > > Konrad Rzepecki: > >> I have similar problem, and I suspect you may also hit it. In my case I > >> send ~10k-20k mails through sendmail wrapper. My problem is caused by > >> fsync done on EVERY queued mail. It take on my servers (

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Konrad Rzepecki
W dniu 18.01.2012 13:03, Wietse Venema pisze: Konrad Rzepecki: I have similar problem, and I suspect you may also hit it. In my case I send ~10k-20k mails through sendmail wrapper. My problem is caused by fsync done on EVERY queued mail. It take on my servers (4 core AMD @ ext4 @ LVM @ RAID10 @

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Konrad Rzepecki
W dniu 18.01.2012 13:07, Wietse Venema pisze: Ralf Hildebrandt: eatmydata postfix start But I don't know if it's LD_PRELOAD trick will work with postfix out of the box. postdrop is setgid, and will not work with LD_PRELOAD. Too bad :( Why are you queuing one file per recipient? That reall

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Wietse Venema
Ralf Hildebrandt: > eatmydata postfix start > > But I don't know if it's LD_PRELOAD trick will work with postfix out > of the box. postdrop is setgid, and will not work with LD_PRELOAD. Why are you queuing one file per recipient? That really sucks performance even without a broken Linux fsync.

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Wietse Venema
Konrad Rzepecki: [ Charset ISO-8859-1 unsupported, converting... ] > W dniu 18.01.2012 10:29, Simone Ruffilli pisze: > > Hello everyone, > > > > whenever I submit to my postfix server a mail having a massive (~15k) > > recipient list, it takes forever to accept and start delivering it. > > I have

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Konrad Rzepecki
W dniu 18.01.2012 12:03, Ralf Hildebrandt pisze: * Konrad Rzepecki: I've already tried SMTP. This make no change since it also create queue file, which must be fsynced before client is notified about successfully queuing... sendmail submission writes two files (maildrop queue). SMTP submissio

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Konrad Rzepecki : > >You can use mini_sendmail (which uses SMTP but gives you a sendmail > >command for that) > > I've already tried SMTP. This make no change since it also create > queue file, which must be fsynced before client is notified about > successfully queuing... sendmail submission

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Konrad Rzepecki
W dniu 18.01.2012 11:50, Ralf Hildebrandt pisze: * Konrad Rzepecki: I have similar problem, and I suspect you may also hit it. In my case I send ~10k-20k mails through sendmail wrapper. My problem is caused by fsync done on EVERY queued mail. It take on my servers (4 core AMD @ ext4 @ LVM @ RAI

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Ralf Hildebrandt : > * Konrad Rzepecki : > > > I have similar problem, and I suspect you may also hit it. In my case > > I send ~10k-20k mails through sendmail wrapper. My problem is caused > > by fsync done on EVERY queued mail. It take on my servers (4 core AMD > > @ ext4 @ LVM @ RAID10 @ 8dis

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Konrad Rzepecki : > I have similar problem, and I suspect you may also hit it. In my case > I send ~10k-20k mails through sendmail wrapper. My problem is caused > by fsync done on EVERY queued mail. It take on my servers (4 core AMD > @ ext4 @ LVM @ RAID10 @ 8disk) ~ 0.2-0.3s for each one. Somet

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Konrad Rzepecki
W dniu 18.01.2012 10:29, Simone Ruffilli pisze: Hello everyone, whenever I submit to my postfix server a mail having a massive (~15k) recipient list, it takes forever to accept and start delivering it. I have similar problem, and I suspect you may also hit it. In my case I send ~10k-20k mails

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Simone Ruffilli : > smtpd_client_restrictions = *sleep 1*, reject_rbl_client > bl.spamcop.net, reject_rbl_client sbl.spamhaus.org, reject_rbl_client > xbl.spamhaus.org Ah DAMN > Can that be somehow responsible of my slowness? I tried to send a > mail to 2k generated non-existant remote addre

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Simone Ruffilli
Il 18/01/2012 11:22, Ralf Hildebrandt ha scritto: * Simone Ruffilli: Each recipient has to be checked against virtual_mailbox_domains and virtual_alias_maps (you don't have virtual_alias_domains set?), since the recipient COULD be one of your own recipients. make sure those queries are fast.

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Simone Ruffilli : > No logs actually (I'll check if Thunderbird provides some useful log). > On my last attempt thunderbird stood stuck for ~20 minutes > ("Connected to smtp_server ..."), then postfix (rightfully so!) > complained 5.1.1 about an unexistant local address (this one deserves > anot

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Simone Ruffilli
Il 18/01/2012 10:35, Ralf Hildebrandt ha scritto: * Simone Ruffilli: > whenever I submit to my postfix server a mail having a massive (~15k) > recipient list, it takes forever to accept and start delivering it. How do you submit the mail? Got some logs? No logs actually (I'll check if Thund

Re: Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Ralf Hildebrandt
* Simone Ruffilli : > Hello everyone, > > whenever I submit to my postfix server a mail having a massive (~15k) > recipient list, it takes forever to accept and start delivering it. How do you submit the mail? Got some logs? > Has it something to do with address validation (which, if I > underst

Postfix very slow accepting a mail having a massive recipient list

2012-01-18 Thread Simone Ruffilli
Hello everyone, whenever I submit to my postfix server a mail having a massive (~15k) recipient list, it takes forever to accept and start delivering it. I've done everything in the checklist (namely DNS caching), but it seems like the problem is not related to a DNS issue: tcpdump-ing port 5