Well, I don't think what I'm describing is complicated semantics. I think I've described general batch operation design and something that is symmetrical the batch_mutate method already on the Cassandra API. You are right, I can solve the problem with further denormalization, and the approach of making individual gets in parallel as described by Brandon will work too. I'll be doing one of these for now. But I think neither is as efficient, and I guess I'm still not sure why the multiget is designed the way it is.
The problem with denormalization is you gotta make multiple row writes in place of one, adding load to the server, adding required physical space and losing atomicity on write operations. I know writes are cheap in cassandra, and you can catch failed writes and retry so these problems are not major, but it still seems clear that having a batch-get that works appropriately is a least a little better... ________________________________________ From: Aaron Morton [aa...@thelastpickle.com] Sent: Tuesday, January 18, 2011 12:55 PM To: user@cassandra.apache.org Subject: Re: please help with multiget I think the general approach is to denormalise data to remove the need for complicated semantics when reading. Aaron On 19/01/2011, at 7:57 AM, Shu Zhang <szh...@mediosystems.com> wrote: > Well, maybe making a batch-get is not anymore efficient on the server side > but without it, you can get bottlenecked on client-server connections and > client resources. If the number of requests you want to batch is on the order > of connections in your pool, then yes, making gets in parallel is as good or > maybe better. But what if you want to batch thousands of requests? > > The server I can scale out, I would want to get my requests there without > needing to wait for connections on my client to free up. > > I just don't really understand the reasoning for designing muliget_slice the > way it is. I still think if you're gonna have a batch-get request > (multiget_slice), you should be able to add to the batch a reasonable number > of ANY corresponding non-batch get requests. And you can't do that... Plus, > it's not symmetrical to the batch-mutate. Is there a good reason for that? > ________________________________________ > From: Brandon Williams [dri...@gmail.com] > Sent: Monday, January 17, 2011 5:09 PM > To: user@cassandra.apache.org > Cc: hector-us...@googlegroups.com > Subject: Re: please help with multiget > > On Mon, Jan 17, 2011 at 6:53 PM, Shu Zhang > <szh...@mediosystems.com<mailto:szh...@mediosystems.com>> wrote: > Here's the method declaration for quick reference: > map<string,list<ColumnOrSuperColumn>> multiget_slice(string keyspace, > list<string> keys, ColumnParent column_parent, SlicePredicate predicate, > ConsistencyLevel consistency_level) > > It looks like you must have the same SlicePredicate for every key in your > batch retrieval, so what are you suppose to do when you need to retrieve > different columns for different keys? > > Issue multiple gets in parallel yourself. Keep in mind that multiget is not > an optimization, in fact, it can work against you when one key exceeds the > rpc timeout, because you get nothing back. > > -Brandon
