> Does anyone have some tips on getting peak performance out of mass
> mailings with qmail.
> We regularly send out newsletters to over 500,000 email address's on
> a weekly/fortnightly basis.
> I've looked through the archives and there are some excellent tips
> but I'm still hoping to push it further as a full mailout takes
> nearly a day.
>
> My current setup is qmail-1.03 with Russell Nelson's big-todo patch
> conf-split is set to 47
> concurrencyremote is set to 240
> We have written a simple perl script that takes the whole mailout and
> pipes it directly to qmail-inject
There is a qmail patch (big concurrency) that will allow running more
remotes. The default kernal in RedHat 6.1 limits you to 512 total processes
and half that per user. I have recompiled the kernel for 2560 total
processes and 2048 per user. See the file
/usr/src/linux/include/linux/tacks.h.
You may also have to
echo 8192 > /proc/sys/fs/file-max
echo 24576 > /proc/sys/fs/inode-max
in order to support large numbers of open files. I just add these lines to
/etc/rc.d/rc.local so they run during boot up.
I can get about 50K emails per hour using 400 remotes. That would take 10
hours with your list. The qmail queue size seems to stabilaze between 10K
and 15K during the run. There is usually between 300-400 remotes running.
Another thing you can try is to comment out the calls to fsync in
qmail-queue.c. I found this dramtically sped up queue but not sending. The
downside is messages will be lost if the box crashes.
>
> The mailout flies with the concurrencyremote being hit after the pipe
> to qmail-inject is closed but it takes a long long long time for the
> qmail-inject process to finish.
>
> Does anyone have any tips on how to analyse the performance
> bottle-necks .. disk / bandwidth etc ( this is a redhat linux 6.1
> box) or tips on a better way of doing this.
I am currently experimenting with bypassing qmail altogether. Using shared
memory I fork off 1000 child processes to chew on a list in parallel. Each
child will call qmail-inject only if its direct attempt fails. My testing
is still rough but it looks like I can get a 5 to 10 fold improvement.
If you are only doing this once a week I doubt you need to go to such
extremes. I don't know what kind of bandwidth you have available but given
your schedule do you really need to compress it down to a short time frame.
