[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694344#comment-13694344
]
Pavel Yaskevich commented on CASSANDRA-5661:
--------------------------------------------
After thinking about this more as LCS produces similar (same) sized files there
is actually no benefit of caching [C]RAR instances. I think we should do
caching only with STCS + expiring as it introduces less memory overhead per
file for [C]RAR.
I also want to share my worries about LCS using 5MB file size with mmap and no
compression: default vm.max_map_count is 65536 (maximum number of memory
mappings per process) which is around 322GB of data, we need to mention
somewhere that if somebody wants to run with mmap + LCS they need to adjust
that setting. Another thing that worries me is that performance of mmap, munmap
and similar + minor/major page faults would degrade logarithmically as memory
mappings are handled by red-black tree and with constant compaction we would be
burning a lot more cpu on tree balancing as dataset grows.
> Discard pooled readers for cold data
> ------------------------------------
>
> Key: CASSANDRA-5661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.2.1
> Reporter: Jonathan Ellis
> Assignee: Pavel Yaskevich
> Fix For: 1.2.7
>
> Attachments: DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled
> RandomAccessReaders are never cleaned up until the SSTableReader is closed.
> So memory use is "the worst case simultaneous RAR we had open for this file,
> forever."
> We should introduce a global limit on how much memory to use for RAR, and
> evict old ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
