Re: Sparse vs dense index

2010-03-17 Thread alex kamil
yep, I'll probably try both I don't think there is anything out there which can beat in-memory db in terms of bulk throughput (e.g http://cs.nyu.edu/cs/faculty/shasha/papers/sigmodpap.pdf) but will see how far we can get with open source tools and using a combination of persistent storage and cach

Re: Sparse vs dense index

2010-03-17 Thread Jonathan Ellis
I guess if you are going to read the full 5MB at once then that makes more sense. But if you are going to slice it or access parts by column name then the other does. On Tue, Mar 16, 2010 at 12:15 PM, alex kamil wrote: > which index structure would fit Cassandra more naturally and perform better

Sparse vs dense index

2010-03-16 Thread alex kamil
which index structure would fit Cassandra more naturally and perform better: 1) a sparse index where in each row there are 100 columns each containing a 5MB data block (under a single column family) or 2) a dense index where each row contains 100 columns with a single 6bytes value (under a single