> That is a good reason for both to be configurable IMO.
index sampling is currently configurable only per node, it would be
better to have it per Keyspace because we are using OLTP like and OLAP
keyspaces in same cluster. OLAP Keyspaces has about 1000x more rows.
But its difficult to estimate index sampling memory until there will be
way to monitor memory used by index sampling
https://issues.apache.org/jira/browse/CASSANDRA-3662 . Java can use
about 10x more memory than raw data for index sample entry - and from
sstable/IndexSummary.java it seems that cassandra is using one big
arrayList with <RowPosition,long>.
on node with 300m rows (small node), it will be 585937 index sample
entries with 512 sampling. lets say 100 bytes per entry this will be 585
MB, bloom filters are 884 MB. With default sampling 128, sampled entries
will use majority of node memory. Index sampling should be reworked like
bloom filters to avoid allocating one large array per sstable. hadoop
mapfile is using sampling 128 by default too and it reads entire mapfile
index into memory.
it should be clearly documented in
http://wiki.apache.org/cassandra/LargeDataSetConsiderations - that bloom
filters + index sampling will be responsible for most memory used by
node. Caching itself has minimal use on large data set used for OLAP.
