Itamar Turner-Trauring wrote:
> On Mon, 2010-04-12 at 14:23 -0400, Glyph Lefkowitz wrote:
>
> > Still, I wouldn't recommend using the subprocess module in a Twisted
> > application, and multiprocessing even less. 'subprocess' uses
> > select(), which means that if you are running processes in a s
On Mon, 2010-04-12 at 14:23 -0400, Glyph Lefkowitz wrote:
> Still, I wouldn't recommend using the subprocess module in a Twisted
> application, and multiprocessing even less. 'subprocess' uses
> select(), which means that if you are running processes in a server
> handling a large number of conne
On 13 April 2010 03:39, Jason J. W. Williams wrote:
> Haven't had any issues yet. Twisted imports occur inside the process
> function. The app was originally written as a purely blocking
> multiprocessing app and rewritten to use Twisted inside the
> sub-processes. It's passed all automated and ha
That all makes sense to me if you're using it to coordinate between
the processes using multiprocessing's mechanisms (which I assume most
do). We just aren't. Once the processes are spawned they do their own
thing without any knowledge of each other. There something I should be
expecting to bite me
On Apr 12, 2010, at 12:06 PM, David Ripton wrote:
> On 2010.04.12 09:39:21 -0600, Jason J. W. Williams wrote:
>> Haven't had any issues yet. Twisted imports occur inside the process
>> function. The app was originally written as a purely blocking
>> multiprocessing app and rewritten to use Twiste
The comment about passing installSignalHandlers=False to reactor.run()
is good to know. Are the signal handlers Twisted installs used by
Twisted for anything besides reactor.spawnProcess() or other tasks
related to sub-process management?
-J
On Mon, Apr 12, 2010 at 10:06 AM, David Ripton wrote:
On 2010.04.12 09:39:21 -0600, Jason J. W. Williams wrote:
> Haven't had any issues yet. Twisted imports occur inside the process
> function. The app was originally written as a purely blocking
> multiprocessing app and rewritten to use Twisted inside the
> sub-processes. It's passed all automated a
Yeah...our code doesn't actually use the pipes. They get fed over AMQP
(txAMQP). We're essentially using multiprocessing as a sub-proc
supervisor to spin-up "processing" engines that consume the AMQP
queues they're configured with.
-J
On Mon, Apr 12, 2010 at 9:49 AM, Phil Mayers wrote:
> On 04/1
On 04/12/2010 04:39 PM, Jason J. W. Williams wrote:
> Haven't had any issues yet. Twisted imports occur inside the process
> function. The app was originally written as a purely blocking
> multiprocessing app and rewritten to use Twisted inside the
> sub-processes. It's passed all automated and han
I will say importing your reactor (from twisted.internet import
reactor) must be done inside the sub-process or you won't get what you
want.
-J
On Mon, Apr 12, 2010 at 9:39 AM, Jason J. W. Williams
wrote:
> Haven't had any issues yet. Twisted imports occur inside the process
> function. The app
On 04/12/2010 01:09 PM, Gabriel Rossetti wrote:
> Hello everyone,
>
> I'd like to run a python function in a new process, like python 2.6's
> multiprocessing does. I wanted to use ProcessProtocol, but it seems to
> only work for executing executables and interacting with them using
> stdin/stdout/o
Haven't had any issues yet. Twisted imports occur inside the process
function. The app was originally written as a purely blocking
multiprocessing app and rewritten to use Twisted inside the
sub-processes. It's passed all automated and hand tests without an
issue. Is there a reason importing Twiste
On 2010.04.12 16:23:54 +0200, Gabriel Rossetti wrote:
> Jason J. W. Williams wrote:
> > You might want to try Python's multiprocessing code, it works very
> > well. I've had pretty good luck using it with Twisted inside the
> > processes too. The multiprocessing library handles the work of setting
Jason J. W. Williams wrote:
> Hi Gabriel,
>
> You might want to try Python's multiprocessing code, it works very
> well. I've had pretty good luck using it with Twisted inside the
> processes too. The multiprocessing library handles the work of setting
> up the send and receive file descriptors.
>
> You don't have to write it to disk - you can use cStringIO.
Sorry, please disregard this - I was thinking about something else.
The way I've done Python process spawning before was to have my code
predefined in a string, and then launched via a ProcessProtocol. You
can do tricks whereby you hav
Hi Gabriel,
You might want to try Python's multiprocessing code, it works very
well. I've had pretty good luck using it with Twisted inside the
processes too. The multiprocessing library handles the work of setting
up the send and receive file descriptors.
-J
On Mon, Apr 12, 2010 at 6:09 AM, Gab
On 01:24 pm, alexan...@quessy.net wrote:
>Hello,
>
>2010/4/12 Reza Lotun :
>>>yes, I'd thought of that, but I can't get the function's code, I
>>>tried
>>>the inspect module but
>>>it only works of the code is written to disk; I'd rather not have to
>>>first write the code to disk just to be able
Hello,
2010/4/12 Reza Lotun :
>> yes, I'd thought of that, but I can't get the function's code, I tried
>> the inspect module but
>> it only works of the code is written to disk; I'd rather not have to
>> first write the code to disk just to be able to turn it into a string
>> and feed it to pytho
> yes, I'd thought of that, but I can't get the function's code, I tried
> the inspect module but
> it only works of the code is written to disk; I'd rather not have to
> first write the code to disk just to be able to turn it into a string
> and feed it to python -c. Any ideas on how I could do th
Reza Lotun wrote:
> Hi,
>
>
>> I'd like to run a python function in a new process, like python 2.6's
>> multiprocessing does. I wanted to use ProcessProtocol, but it seems to
>> only work for executing executables and interacting with them using
>> stdin/stdout/other fds. Is there a way of runni
Hi,
> I'd like to run a python function in a new process, like python 2.6's
> multiprocessing does. I wanted to use ProcessProtocol, but it seems to
> only work for executing executables and interacting with them using
> stdin/stdout/other fds. Is there a way of running a python piece of code
> in
Hello everyone,
I'd like to run a python function in a new process, like python 2.6's
multiprocessing does. I wanted to use ProcessProtocol, but it seems to
only work for executing executables and interacting with them using
stdin/stdout/other fds. Is there a way of running a python piece of co
22 matches
Mail list logo
