Re: [Twisted-Python] Twisted in a multicore environment

2012-07-15 Thread gelin yan
On Sun, Jul 15, 2012 at 1:38 AM, Werner Thie wrote: > On 7/14/12 12:39 AM, gelin yan wrote: > > > > > > On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie > > wrote: > > > > On 7/10/12 6:17 AM, Laurens Van Houtven wrote: > > > FWIW, I have used Ampoule to great

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-14 Thread Werner Thie
On 7/14/12 12:39 AM, gelin yan wrote: > > > On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie > wrote: > > On 7/10/12 6:17 AM, Laurens Van Houtven wrote: > > FWIW, I have used Ampoule to great effect, but as JP points out > it's hardly the only option. You'r

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-14 Thread gelin yan
On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie wrote: > On 7/10/12 6:17 AM, Laurens Van Houtven wrote: > > FWIW, I have used Ampoule to great effect, but as JP points out it's > hardly the only option. You're bound to end up with some measure of > multiprocessing. Bear in mind that not all workloa

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-12 Thread Ray Cote
Hi Augusto: You don't mention the protocol you're implementing in Twisted, but if is is HTTP... The way we address this problem with an HTTP protocol is to put multiple copies of Twisted behind nginx. On two and four core machines, we set the number of nginx worker processes equal to the nu

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-12 Thread Corbin Simpson
On Thu, Jul 12, 2012 at 08:10:45AM +0200, Vincent Bernat wrote: > Is there something similar to Ampoule but allowing to spawn processes > across different servers? Something that would provide > "deferToRemoteAMPProcess". AFAIK, not exactly. You can pretty easily craft this yourself, though. The

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-11 Thread Vincent Bernat
❦ 10 juillet 2012 20:08 CEST, Werner Thie  : > We observed really great scaling on multi cores with moving the > application part either to ampoule for PDF production or in the other > case I wrote an implementation of self regulating process pool based on > spread, leaving only the serving to

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-10 Thread Werner Thie
On 7/10/12 6:17 AM, Laurens Van Houtven wrote: > FWIW, I have used Ampoule to great effect, but as JP points out it's hardly > the only option. You're bound to end up with some measure of multiprocessing. > Bear in mind that not all workloads are well-suited for this kind of problem! > Always m