Re: memory estimate for each key in the key cache

2011-12-20 Thread aaron morton
If you get the sizing wrong there area couple of emergency pressure valves in the config… https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L113 Watch for log messages with "Cassandra is now reducing cache sizes to free up memory." in them. Cheers - Aaron Mort

Re: memory estimate for each key in the key cache

2011-12-20 Thread Kent Tong
> It is not telling you to multiply your key size by 10-12, it is telling you to > multiply the output of the nodetool cfstats reported "key cache size" by > 10-12. The  "key cache size" reported is actually the number of keys in the key cache. So, it is the same thing as suggesting each key ta

Re: memory estimate for each key in the key cache

2011-12-19 Thread Jeremiah Jordan
It is not telling you to multiply your key size by 10-12, it is telling you to multiply the output of the nodetool cfstats reported "key cache size" by 10-12. -Jeremiah On Dec 18, 2011, at 6:37 PM, Guy Incognito wrote: > to be blunt, this doesn't sound right to me, unless it's doing something

Re: memory estimate for each key in the key cache

2011-12-18 Thread Guy Incognito
to be blunt, this doesn't sound right to me, unless it's doing something rather more clever to manage the memory. i mocked up a simple class containing a byte[], ByteBuffer and long, and the shallow size alone is 32 bytes. deep size with a byte[16], 1-byte bytebuffer and long is 132. this is

Re: memory estimate for each key in the key cache

2011-12-16 Thread Brandon Williams
On Fri, Dec 16, 2011 at 9:31 PM, Dave Brosius wrote: > Wow, Java is a lot better than I thought if it can perform that kind of > magic.  I'm guessing the wiki information is just old and out of date. It's > probably more like 60 + sizeof(key) With jamm and MAT it's fairly easy to test. The numbe

Re: memory estimate for each key in the key cache

2011-12-16 Thread Dave Brosius
On 12/16/2011 10:13 PM, Brandon Williams wrote: On Fri, Dec 16, 2011 at 8:52 PM, Kent Tong wrote: Hi, From the source code I can see that for each key, the hash (token), the key itself (ByteBuffer) and the position (long. offset in the sstable) are stored into the key cache. The hash is an

Re: memory estimate for each key in the key cache

2011-12-16 Thread Brandon Williams
On Fri, Dec 16, 2011 at 8:52 PM, Kent Tong wrote: > Hi, > > From the source code I can see that for each key, the hash (token), the key > itself (ByteBuffer) and the position (long. offset in the sstable) are stored > into the key cache. The hash is an MD5 hash, so it is 16 bytes. So, the total

memory estimate for each key in the key cache

2011-12-16 Thread Kent Tong
Hi, From the source code I can see that for each key, the hash (token), the key itself (ByteBuffer) and the position (long. offset in the sstable) are stored into the key cache. The hash is an MD5 hash, so it is 16 bytes. So, the total size required is at least 16+size-of(key)+4 which is > 20 b