On Thu, Jul 8, 2010 at 12:18 AM, Terje Marthinussen <tmarthinus...@gmail.com> wrote: > 1. There is just too much code and too many "layers" involved when hasNext > is called. I suspect this requires a re-design and the > google/common-collections may have to be thrown out. This would seem to be a > pretty critical area of Cassandra code for performance, so I suspect the > cost of making functionality specifically tuned for Cassandra should be > worth it.
That might be interesting. I've looked at the apache collated iterator code [the main thing we use commons-collections for] and it looks pretty sloppy. Google collections (guava in trunk) otoh is probably better than what I could come up with. Kevin from guava suggests they will add support for collated iteration "eventually." See http://code.google.com/p/guava-libraries/issues/detail?id=119 and the linked 220. > 2. From what I understand, Columns from all SS/MemTables are merged before > considering tombstones and applying the slice predicates. Partly -- for slicing, we deserialize an index block at a time (from each sstable) until the the slice predicate is satisfied (getNextBlock in ColumnGroupReader). For reversed queries this is unavoidable, but it could be optimized better for forward queries. For column name predicates we deserialize exactly the requested columns. It should be possible to optimize both better wrt tombstones (instead of throwing out "uninteresting" columns in a final pass, pass in the deletion time they have to "beat" to be relevant to the IColumnIterators). > While this may make things somewhat hairy codewise, JVM performance/memory > consumption tend to not be happy in scenarios with heavy object creation, > and we get a lot of objects here. Agreed. > 3. It would seem like there might be good possibilities to do a fair bit of > optimization for the case where a complete set of columns are fetched > (slices with null for start/end predicates), but I am not currently sure how > much time it would save and if it would be worth the work and added > complexity to the As I commented on https://issues.apache.org/jira/browse/CASSANDRA-1207, one piece of low-hanging fruit would be to allow skipping either the column index or bloom filter for name-based or slice-based queries, respectively. > I may make an attempt at 1. but I don't feel entirely like I understand > enough of the Cassandra code yet to do any of the above at the moment so it > may take a little while. Feel free to catch us on IRC in #cassandra-dev if you want quicker answers. :) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
