I know the multiprocessing module is not properly supported by twisted apps
because of the interactions among duplicated file descriptors and signal
handling, as discussed other times.
But python 3.4 introduces a new mode to use that module by spawning (i.e.
fork() followed by execv()) the new processes instead of simply forking it.
So my question is how supported this is by twisted, and in general how safe it
is to use subprocesses created by duplicating the parent immediately followed
by the execv of a fresh interpreter.
What i'm thinking is something like this, to asynchronously process requests
and delegate the cpu-bound work to some processes:
import multiprocessing
def worker(q):
while True:
work = q.get()
do_something_cpu_bound_with(work)
def main():
context = multiprocessing.get_context('spawn')
q = context.Queue()
p = context.Process(target=worker, args=(q,))
p.start()
q.put_nowait(work) # when async requests are made
if __name__ == '__main__':
from twisted.internet import reactor
reactor.callWhenRunning(main)
reactor.run()
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
