On Jun 12, 2012, at 1:53 PM, Quan Nguyen wrote: > Hello, > > I am new to Twisted. > I am extending the twisted Echo server example by sending the data that the > server receives from telnet into a child process that processes the data. The > function outReceived in censoringProtocol is called when the child writes the > result and the result is received. The process of passing data from parent to > child and back to parent is done by having a while loop of stdin.readline() > and stdout.write() in the child process. > Specifically, I generate a deferred object each time I write to the child > using the function sendData (which then calls writeToChild) and use a queue > to manage the deferred objects. I add a callBack (printData) that will > broadcast the result to all instances of Echo when outReceived is called. > > I have two questions: > 1/ Is this the right approach to communicate with the child process in order > to achieve what I want? Are there any gotcha's?
This sounds pretty much fine, except for the caveat you already asked about: > 2/ This server will be exposed to high loads. There will be many instances of > Echo sending data to the child at a high rate. Are there any limitations that > I have to watch for, such as the limit on child process's pipe sizes or > anything else? You might want to investigate the producer/consumer API, as recently blogged about by our marvelously productive summer intern, ashfall: <http://ashfall.github.com/blog/2012/05/29/twisted-producer-and-consumer-system/>. As currently written, even if your subprocess backs up, you will buffer data to it without any limits. (The consumer you want, by the way, is censoringProtocol().transport.pipes[0], in case that isn't entirely clear; processes need to be documented a little better.) You may have to write a little bit of tricky code to multiplex the consumer, if you want to have multiple "echo" client connections using the same subprocess. If it were a 1<->1 relationship you could just register the echo protocol's transport as the producer for the stdin pipe on the processing protocol, but spawning that many processes is probably wasteful. -glyph _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
