Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-12 Thread Tyler Hobbs
Well, in this case I would say you probably need about 300MB of space in the heap, since that's what you've calculated. The APIs are designed to let you do what you think is best and they definitely won't stop you from shooting yourself in the foot. Counting a huge row, or trying to grab every ro

Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-12 Thread Dave Martin
Thanks Tyler. I was unaware of counters. The use case for column counts is really from a operational perspective, to allow a sysadmin to do adhoc checks on columns to see if something has gone wrong in software outside of cassandra. I think running a cassandra-cli command such as count, which mak

Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-11 Thread Tyler Hobbs
Cassandra has to deserialize all of the columns in the row for get_count(). So from Cassandra's perspective, it's almost as much work as getting the entire row, it just doesn't have to send everything back over the network. If you're frequently counting 8 million columns (or really, anything signi