On Mon, Sep 12, 2011 at 11:00 AM, Jon Harris <j.har...@digital-ink.co.uk> wrote: > Hi Mark > > I know this does sound odd, but the middleware uses its SQLite and its > own SMTP "queue management".
Bypass it. Tell it that the smart relay host is your postfix box. It is trying to do something it shouldn't. Postfix should be the only software that manages queues. Apparently when the queue gets bloated, > things start to go wrong. Whereas creating 72,000 files on disk wouldn't > be a problem. Yes, I agree this does expose a weekness in the > middleware, but we are already committed to it. Most web applications > only have to email an order or a contact form, etc. So I can't really > blame the developers for me trying to use it as a mailing list manager. > > So, my idea was to use something a little more "industrial strength" to > get the messages out. Yes, it is called postfix. > > On 12/09/2011 14:37, Jon Harris wrote: >> Hi List >> >> I don't know if this is possible.... >> >> But we have developed a website for a customer with a CRM backend, >> without around 72,000 subscribers. >> >> We want to update and manage the subscriber list on the webserver and >> generate the weekly mailshot, all these processes are running fine. >> >> Our solution was to create threads on the web server (running IIS) to >> generate emails to be sent by the middleware (a PHP like language). >> This does work fine up to around a couple of thousand emails, then it >> runs out of memory and the process dies. I am looking at some kind of >> throttle, but its not a very elegant solution. >> >> I thought if I could generate a postfix friendly file, I could create >> an SMB share and drop the 72,000 files into a folder that Postfix >> would "see" and then process. > > I may be missing something obvious here, but I don't see why a program > that can't create 72,000 email messages without running out of memory > will be able to create 72,000 files without failing for exactly the same > reason. An email is, after all, merely a file which is written to an > external process (an SMTP server, in this example example) rather than a > physical location. In fact, I'm having difficulty imagining a reason why > any competently written program can't send an arbitrarily large number > of emails anyway - all it's doing is running round a loop repeatedly, > and it doesn't matter whether it does it 72 times or 72,000 times. The > real constraints are the capacity of your mail servers to handle that > many emails. > > It sounds to me, therefore, is if what you've got is a serious flaw in > your mail-generation program, and I'd be inclined to address that rather > than trying to find a workaround for the problem that involves injecting > mail directly into the queue. > > If you really can't fix the mail-generation program's sending limit (eg, > because it's a third-party CMS that comes with the bug built-in), then, > as has already been suggested, the simplest option is to use an > intermediate program which takes files from your storage directory and > then sends them out by email. That's not Postfix-specific; any program > which can read a file and then write to SMTP will work with any MTA. It > would be pretty trivial to code that in PHP or Perl, I'm presuming it > would be equally simple in Python, Ruby, ASP or your programming > language of choice. The only complex part of it is error handling to > ensure that you don't send the same message twice to the same person or > that you don't miss anyone out if any email fails to send. > > Mark > -- > Sent from my Babbage Difference Engine > http://mark.goodge.co.uk > http://www.ratemysupermarket.com >
