Re: Cassandra 1.2 TTL histogram problem

2013-05-23 Thread Yuki Morishita
> Are you sure that it is a good idea to estimate remainingKeys like that? Since we don't want to scan every row to check overlap and cause heavy IO automatically, the method can only do the best-effort type of calculation. In your case, try running user defined compaction on that sstable file. It

Re: Cassandra 1.2 TTL histogram problem

2013-05-22 Thread cem
Thanks for the answer. It means that if we use randompartioner it will be very difficult to find a sstable without any overlap. Let me give you an example from my test. I have ~50 sstables in total and an sstable with droppable ratio 0.9. I use GUID for key and only insert (no update -delete) s

Re: Cassandra 1.2 TTL histogram problem

2013-05-22 Thread Yuki Morishita
> Can method calculate non-overlapping keys as overlapping? Yes. And randomized keys don't matter here since sstables are sorted by "token" calculated from key by your partitioner, and the method uses sstable's min/max token to estimate overlap. On Tue, May 21, 2013 at 4:43 PM, cem wrote: > Than

Re: Cassandra 1.2 TTL histogram problem

2013-05-21 Thread cem
Thank you very much for the swift answer. I have one more question about the second part. Can method calculate non-overlapping keys as overlapping? I mean it uses max and min tokens and column count. They can be very close to each other if random keys are used. In my use case I generate a GUID fo

Re: Cassandra 1.2 TTL histogram problem

2013-05-21 Thread Yuki Morishita
> Why does Cassandra single table compaction skips the keys that are in the > other sstables? because we don't want to resurrect deleted columns. Say, sstable A has the column with timestamp 1, and sstable B has the same column which deleted at timestamp 2. Then if we purge that column only from

Cassandra 1.2 TTL histogram problem

2013-05-21 Thread cem
Hi all, I have a question about ticket https://issues.apache.org/jira/browse/CASSANDRA-3442 Why does Cassandra single table compaction skips the keys that are in the other sstables? Please correct if I am wrong. I also dont understand why we have this line in worthDroppingTombstones method: dou