Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Brad Milne
Thanks all for the thoughts. Clinching drivers for us to choose AMQP (RabbitMQ) include built-in persistence, manual ACK (ie only when finished, not just when pulled), plus the obviously strong configuration-driven routing capabilities. With this distributed architecture using rabbit, we're not l

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Glyph
On Oct 20, 2011, at 6:50 AM, Brad Milne wrote: > We have a core message-handling platform which encompasses an ever-increasing > array of modules and services run inside one Twisted process. This is good as > it means the reactor is in control of most of the stuff relevant to it. > However, as

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Itamar Turner-Trauring
> Can you list some of the things that might go wrong? I've used rebuild in > production, and it's perfectly fine if you understand the inherent > limitations of Python's object model reloading code. For services with > long startup time and lots of data in core it can be the best option. You mi

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Glyph
On Oct 20, 2011, at 7:50 AM, Itamar Turner-Trauring wrote: > On Thu, 2011-10-20 at 12:29 +0100, Reza Lotun wrote > >> 1. http://twistedmatrix.com/documents/current/api/twisted.python.rebuild.html >> This a module that can be used to reload class definitions and >> instances, etc. > > This is pro

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Christopher Armstrong
On Thu, Oct 20, 2011 at 6:50 AM, Brad Milne wrote: > 2) Is RabbitMQ with txAMQP-empowered Twisted instances on the ends of the > queues a sensible, compatible, best-practice approach as agreed by the > community? > Yeah, do this. One of these days I'm going to release a library for making deplo

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Itamar Turner-Trauring
On Thu, 2011-10-20 at 23:50 +1300, Brad Milne wrote: > To date we have begun introducing RabbitMQ (AMQP), with separate > Twisted processes running as adapters at either end of queues. On the > surface this fits nicely as it provides distributed architecture, > resilience through isolation, and de

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Itamar Turner-Trauring
On Thu, 2011-10-20 at 12:29 +0100, Reza Lotun wrote > 1. http://twistedmatrix.com/documents/current/api/twisted.python.rebuild.html > This a module that can be used to reload class definitions and > instances, etc. This is probably not something you want to use in production. Too many thing can g

Re: [Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Reza Lotun
Hi Brad, On Thu, Oct 20, 2011 at 11:50 AM, Brad Milne wrote: > ... > So my questions: > 1) I'm right, right? Twisted doesn't have mechanisms or a library for hot > swapping code modules or similar solutions for low impact upgrades and high > availability? (I originally read about Twisted *plugins

[Twisted-Python] Twisted enterprise deployment models

2011-10-20 Thread Brad Milne
We have a core message-handling platform which encompasses an ever-increasing array of modules and services run inside one Twisted process. This is good as it means the reactor is in control of most of the stuff relevant to it. However, as the number of deployed services within it grows, it becomes