You are correct, it is not a queue in the classic sense... I'm storing the entire "conversation" with a client in perpetuity, and then playing it back in the order received.
Rabbitmq/activemq etc all have about the same throughput 3-6K persistent messages/sec, and are not good for storing the conversation forever... Also I can easily scale cassandra past that message rate and not have to worry about which message broker/cluster I'm connecting to/has the conversation/etc. On Thu, Apr 1, 2010 at 7:02 PM, Keith Thornhill <ke...@raptr.com> wrote: > you mention never deleting from the queue, so what purpose is this > serving? (if you don't pop off the front, is it really a queue?) > > seems if guaranteed order of messages is required, there are many > other projects which are focused towards that problem (rabbitmq, > kestrel, activemq, etc) > > or am i misunderstanding your needs here? > > -keith > > On Thu, Apr 1, 2010 at 6:32 PM, Jeremy Davis > <jerdavis.cassan...@gmail.com> wrote: > > I'm in the process of implementing a Totally Ordered Queue in Cassandra, > and > > wanted to bounce my ideas off the list and also see if there are any > other > > suggestions. > > > > I've come up with an external source of ID's that are always increasing > (but > > not monotonic), and I've also used external synchronization to ensure > only > > one writer to a given queue. And I handle de-duping in the app. > > > > > > My current solution is : (simplified) > > > > Use the "QueueId", to Key into a row of a CF. > > Then, every column in that CF corresponds to a new entry in the Queue, > with > > a custom Comparator to sort the columns by my external ID that is always > > increasing. > > > > Technically I never delete data from the Queue, and I just page through > it > > from a given ID using a SliceRange, etc. > > > > Obviously the problem being that the row needs to get compacted. so then > I > > started bucketizing with multiple rows for a given queue (for example one > > per day (again I'm simplifying))...(so the Key is now "QueueId+Day"...) > > > > Does this seem reasonable? It's solvable, but is starting to seem > > complicated to implement... It would be very easy if I didn't have to > have > > multiple buckets.. > > > > > > > > My other thought is to store one entry per row, and perform > get_range_slices > > and specify a KeyRange, with the OrderPreservingPartitioner. > > But it isn't exactly clear to me what the Order of the keys are in this > > system, so I don't know how to construct my key and queries > appropriately... > > Is this Lexical String Order? Or? > > > > So for example.. Assuming my QueueId's are longs, and my ID's are also > > longs.. My key would be (in Java): > > > > long queueId; > > long msgId; > > > > key = "" + queueId + ":" + msgId; > > > > And if I wanted to do a query my key range might be from > > start = "" + queueId + ":0" > > end = "" + queueId + ":" + Long.MAX_VALUE; > > > > (Will I have to left pad the msgIds with 0's)? > > > > And is this going to be efficient if my msgId isn't monotonically > > increasing? > > > > Thanks, > > -JD > > > > > > > > > > > > > > > > > > > > > > > > > > >
