On 6/11/09, dan trainor <dan.trai...@gmail.com> wrote: > > > > On Thu, Jun 11, 2009 at 1:32 PM, Wietse Venema <wie...@porcupine.org>wrote: > >> dan trainor: >> > Hello, all - >> > >> > I've sent an email through Postfix which has one recipient, which is an >> > alias via alias_maps (mysql lookup table). I've had just a little bit >> of >> > experience with this type of delivery, but not a lot of experience with >> this >> > many final recipients. >> > >> > Right now I see the message sitting in the 'active' queue, but its been >> > sitting for some time. >> >> Is this before or after alias expansion? It can take some time to >> expand 300k aliases from SQL. In fact, the local delivery agent >> may be terminated by a watchdog timer (daemon_timeout = 18000s). >> >> I suspect that SQL is taking its time. >> >> A minor concern: the expansion of 300k aliases will be written to >> a new queue file, so it needs to fit within the message_size_limit >> setting. >> >> Once the new queue file is complete, the queue manager will >> be quite busy scheduling deliveries. >> >> You may want to dry-run test this without outgoing mail enabled. >> >> Wietse >> > > Good evening, Wietse - > > Shortly after sending that message out, I realized that this was > happening. I do in fact see the 'local' transport very busy, eating as much > CPU as it can muster. > > The message sitting in the queue has been placed there before alias > expansion I would assume. I say that assuming the sleeping thread in MySQL > from the Postfix map resolution process is the result of already having > queried MySQL, and also that there is only one message in that 'active' > queue which does not have those aliases expanded. > > I'm going to give it some more time, and see what happens first - the > 'local' transport finishes up as the result of the message(s) being sent, > being killed by daemon_timeout, or something else. > > That's a good point regarding message size limit; I did not think about > that. This would clearly be the cumulation of those final recipient > addresses in the message itself, as documented by "...including envelope > information." > > Thanks again for your time. > > Thanks! > -dant > > Hi -
Just to follow up.... looks like this process has taken too long. I eventually killed it. I'm happy that things are working *exactly* as they should, however. We ended up splitting up that list of 300,000+ recips in to around 6 aliases of 50,000 recips. This method is/was exponentially faster. I think at this point we're going to consider a MySQL VIEW to convert one of those aliases in to a more normalized group of recipients so we can automate breaking these messages up in the future. Thanks again, Wietse - I sincerely appreciate the input. Thanks! -dant
