@aaron - thanks a lot. i will test it. This is very much needed. Cheers, Deepu.
On Tue, Jul 27, 2010 at 6:03 PM, aaron morton <aa...@thelastpickle.com>wrote: > Today I worked out how to make non blocking calls to Cassandra inside of > the non blocking Tornado web server (http://www.tornadoweb.org/) using > Python. I thought I'd share it here and see if anyone thinks I'm abusing > Thrift too much and inviting trouble. > > It's a bit mucky and I have not tested it for things like timeouts and > errors. But here goes... > > The idea is rather than calling a cassandra client function like > get_slice(), call the send_get_slice() then have a non blocking wait on the > socket thrift is using, then call recv_get_slice(). > > So the steps in Tornado are: > > 1. Web handler creates an object from the model, calls a function on it > like start_read() to populate it. > > 2. model.start_read() needs to call get_slice() on the thrift generated > Cassandra client. Instead it calls send_get_slice() and returns to the > calling web handler. > > 3 Web Handler then asks Tornado to epoll wait for any activity on the > thrift socket. It gets access to the socket file descriptor by following > this chain from the thrift generated Cassandra client > _iprot.trans.__TTransportBase__trans.handle.fileno() > > 4, Web handler function called in 1 above returns, Tornado keeps the http > connection alive and the web handler instance alive. Later when the socket > has activity Tornado will call back into the web handler. > > 5. To get the result of the call to cassandra the Web Handler calls a > function on the model such as finish_read(). finish_read() wants to get the > results of the get_slice() and do something, so it calls recv_get_slice on > the thrift Cassandra client. Processes the result and returns to the web > handler. > > > This looks like the same process the TTwisted.py transport in the thrift > package is using. Except it's not using the nasty reference to get to the > raw socket. > > I'm not sure about any adverse affects on the Cassandra server from the > client not servicing the socket immediately when it starts sending data > back. I'm guessing there are some buffers there, but not sure. Could I be > accidentally blocking / hurting the cassandra server ? > > Thanks > Aaron >
