----- Original Message -----
From: "Victor Duchovni" <victor.ducho...@morganstanley.com>
To: <postfix-users@postfix.org>
Sent: Monday, March 23, 2009 4:08 PM
Subject: Re: Mail drop
On Mon, Mar 23, 2009 at 03:58:29PM -0400, Brandon Hilkert wrote:
Having said that, we build a huge text file (~30GB) with about 1 million
eml messages as its contents. The sender utility then parses out email by
email and submits it to the IIS SMTP. We're trying to not have to modify
the sender that much, so I was wondering if I could write a comparable
perl
script to do the same or something like that. So maybe something that I
could submit a text string to (containing the message and all the
headers)
and have it be submitted to postfix.
There's your bottle-neck. The 30GB file created and read sequentially.
This is a terrible design, and with this in place nothing you can do
will make the process faster.
The right approach is to store one copy of the message as a template,
use a pool of processes or threads to send messages in parallel, and
to use a database back-end to retrieve users for processing and mark
them done (recording the VERP id for each recipient so you can process
bounces).
I appreciate the insight. Unfortunately the process is what it is. I don't
have any control over development. My job is to make sure the systems work
properly. I'm trying to help as asked. The file also contains html to allow
a user to see the contents in a browser if they choose. Point being, the
idea of redesigning the system isn't up for debate.
If you are filling a bottle through a straw, having a wider bottle
won't help.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majord...@postfix.org?body=unsubscribe%20postfix-users>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
