On Thu, 2011-03-31 at 13:56 +0200, Bjorn Borud wrote: > > (Hopefully )for the next version, we'll replace Thrift with a > > dedicated protocol, one that eliminates the Thrift dependency, and > > more importantly, implements streaming. This should be transparent > > to applications for the most part though. > > pardon my ignorance, is there a JIRA issue for this or a wiki page > that describes this? I'd be very interested in reading up a bit about > this.
There isn't an issue yet, no. Rackspace (my employer), is scaling back its involvement with Cassandra post-0.8. I'm not yet sure what that will mean, but I'll almost certainly have less time to spend on this (worst-case none at all). So, I have a running list of thoughts/plans I plan to dump into JIRA before I become scarce, and this one is high on my list. However, if this is something you have an interest in, don't feel like you need to wait on me. Feel free to submit a ticket and get started! > I've been using Netty lately to implement a protocol for streaming > messages and I've had quite good results. I used the varint framing > codec that comes with Netty in conjunction with protobuffers. the > performance is quite good. on my laptop I was able to push 400k+ > messages per second from the client to the server on a warm JVM and > about 350k messages per second on a cold JVM. cursory measurements > seem to indicate that serializing protobuffer messages was more > expensive than de-serializing them. I know it sounds like NIH, but I'd like to see us roll our own serialization. It's really not as much work as it sounds; It amounts to a small subset of a generalized framework like PB or Thrift. I'm of the opinion that the benefits derived from specialization, and the reduced external dependencies makes it worth it here. > I like the Netty API. I was going to use my own, minimal NIO library > (just a simple implementation of the Reactor Pattern), but after > playing with Netty for a bit I decided that I'd be much better off > using that. > > a bonus is that adding encryption, compression etc to a pipeline > appears to be relatively easy with Netty. I think it would make sense > to just implement the new protocol as a Netty codec. > > (Originally I tried Apache MINA, but I got a tip from a colleague that > development on Netty is more active. I haven't really compared the > two projects thoroughly enough to have an opinion on this). Full ACK; I have some recent experience with Netty as well and quite like it. -- Eric Evans eev...@rackspace.com
