[
https://issues.apache.org/jira/browse/SOLR-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371616#comment-17371616
]
Mark Robert Miller commented on SOLR-15510:
-------------------------------------------
Almost all large scale data structures can be done off heap.
Often, the focus here is on IO touching items, as the biggest win with Java off
heap is generally moving around bytes from outside the heap to outside the heap.
There is generally less focus here for other large data structures, largely
because allocating offheap memory is expensive, ByteBuffers in Java have had a
variety of historical Java version perf and gc / mem related issues, on heap
byte array performance, etc.
For large scale applications, often sitting on huge amounts of RAM, many
systems offer or prefer or force most large data structures off heap due to the
benefits around garbage collection and memory management, and tradeoff
reevaluations at large scale.
A basic byte buffer cache can help to some degree, optional more aggressive
implementations and memory management can help in others, and in some cases a
scalability, reliability, workload, other gains, tradeoff inversion can justify
various optional configurations targeting different performance/resource usage
profiles.
> Add additional off heap data structure options and ByteBuffer caching for
> reduced GC at scale.
> ----------------------------------------------------------------------------------------------
>
> Key: SOLR-15510
> URL: https://issues.apache.org/jira/browse/SOLR-15510
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Mark Robert Miller
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
