On Mon, Mar 23, 2009 at 04:16:06PM -0400, Brandon Hilkert wrote:
>>> Having said that, we build a huge text file (~30GB) with about 1 million
>>> eml messages as its contents. The sender utility then parses out email by
>>> email and submits it to the IIS SMTP. We're trying to not have to modify
>>> the sender that much, so I was wondering if I could write a comparable
>>> perl script to do the same or something like that. So maybe something that
>>> I could submit a text string to (containing the message and all the
>>> headers) and have it be submitted to postfix.
>>
>> The right approach is to store one copy of the message as a template,
>> use a pool of processes or threads to send messages in parallel, and
>> to use a database back-end to retrieve users for processing and mark
>> them done (recording the VERP id for each recipient so you can process
>> bounces).
>>
>
> I appreciate the insight. Unfortunately the process is what it is. I don't
> have any control over development. My job is to make sure the systems work
> properly. I'm trying to help as asked. The file also contains html to allow
> a user to see the contents in a browser if they choose. Point being, the
> idea of redesigning the system isn't up for debate.
>
The application won't run any faster than the code that serially parses
the 30GB file. If this code can use a pool of SMTP sender "threads" and
can parse the file quickly enough, you could try that.
>> If you are filling a bottle through a straw, having a wider bottle
>> won't help.
You need to make submission parallel. Good luck.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majord...@postfix.org?body=unsubscribe%20postfix-users>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
