On Mon, Mar 23, 2009 at 03:58:29PM -0400, Brandon Hilkert wrote:
> Having said that, we build a huge text file (~30GB) with about 1 million
> eml messages as its contents. The sender utility then parses out email by
> email and submits it to the IIS SMTP. We're trying to not have to modify
> the sender that much, so I was wondering if I could write a comparable perl
> script to do the same or something like that. So maybe something that I
> could submit a text string to (containing the message and all the headers)
> and have it be submitted to postfix.
There's your bottle-neck. The 30GB file created and read sequentially.
This is a terrible design, and with this in place nothing you can do
will make the process faster.
The right approach is to store one copy of the message as a template,
use a pool of processes or threads to send messages in parallel, and
to use a database back-end to retrieve users for processing and mark
them done (recording the VERP id for each recipient so you can process
bounces).
If you are filling a bottle through a straw, having a wider bottle
won't help.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majord...@postfix.org?body=unsubscribe%20postfix-users>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
