On Mon, May 16, 2011 at 2:59 PM, Jeroen Vermeulen <j...@canonical.com> wrote: > On 2011-05-14 14:19, Robert Collins wrote: > >> I believe 2pc is possible with amqp 0.10, but rabbitmq does 0.8 - and >> see the debate around 0mq for > > Something went missing here!
Huh. So anyhow - 0mq is an attempt to build an even faster message substrate than rabbit (which is no slouch) by ditching everything, and then building redundancy etc on top of the super fast substrate. >>> I imagine both problems go away if every message carries a database >>> transaction id, and the job runner keeps an eye on the database >>> transaction >>> log: the runner shouldn't consume a job until the producing transaction >>> has >>> committed, and it should drop jobs whose producers have aborted. Is >>> something along those lines built in? >> >> No, and that sounds terrifying to me - if we tie things up that >> tightly, we may as well just have a queue in the db and poll it every >> 50ms. > > That would be much more costly in both computational and development terms. > But what makes the check for transaction completion sound so scary to you? The check - not scary. The coupling involved - very scary. There are three angles - firstly I can see /some/ messages and queue consumers needing such a sync-and-check, but not all. So s/every/appropriate/ and that would be less scary. Secondly I'd like to be isolating our systems from the db more, rather than building in deeper connections - because our existing deep connections are regularly giving us grief (with issues related to stale transactions, resource management, failover logic, performance (scale up vs scale out)). Lastly we'd be taking messages from a (optionally) persistent queue and having to repersist them until they could be handled (and also deal with the question of whether to do them out-of-order). I think a better model is to only send a message when we're happy for it to be dealt with. E.g. using pg_amqp to send during the commit. -Rob _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
