What you refer to is different than what I need. The real 'Process' implementation is new to Python 2.6 http://docs.python.org/library/multiprocessing.html and is not supported in Twisted at the moment. The Process or threads in Twisted now, use Python threading/process constructs outside of the new multiprocessing module, will suffer from the Python GIL limitations - which hinders higher performance computing. It works, sure. But its not what I'm asking about.
However, I found a workaround for now. Ideally, what I wanted to do was use something like threads.deferToPool() or similar, that _would_ use Python's new support for OS processes. Someone wrote a wrapper to this on the net, but I was curious if this will be supported in Twisted. Darren On Fri, 2010-02-26 at 01:13 +0000, Johann Borck wrote: > Darren Govoni wrote: > > The nice thing about using Python's process support is that you can > > spawn native Processes that run in separate heaps directly from ONE > > Python Twisted app. Not many running side-by-side, which adds the > > complexity of now coordinating among them (however easy with > > additional protocols like spread). > > > > Inter-Process communication is also supported in Python's new > > multiprocessing package. And again, it can all be orchestrated from a > > single service _instance_. > > > > In my code, I need to run "on the metal" for some tasks and not > > others. Agreed, the event-based reactor threading in Twisted is great. > > But not for all modes of computation. For those, I offload onto OS > > processes directly onto CPU cores. Twisted does not provide a way to > > leverage its API against Python's support for this feature. > But twisted provides this feature *itself*, and has done so long before > the multiprocessing module existed - look at Process Protocol > http://twistedmatrix.com/documents/current/core/howto/process.html and > the stdio stuff: > http://twistedmatrix.com/documents/current/core/examples/stdiodemo.py, > or, as has been mentioned before, ampoule. Of course, if you get > multiprocess to work with twisted, that's fine, but you're probably > unnecessarily adding complexity to your application while substracting > compatibility with python versions before 2.6 for no good reason, at > least none you mentioned so far. > > regards, > Johann > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
