[
https://issues.apache.org/jira/browse/SOLR-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868121#comment-17868121
]
Alexander Veit commented on SOLR-17376:
---------------------------------------
I don't think that this does apply here.
Almost all of the indexing work is done by the same thread. The increase in
direct memory allocation shown in the graph above is all caused by this one
thread. As far as I can see the mechanism for the allocation of temporary
direct ByteBuffers cannot explain the verly large number of DirectByteBuffers
under these circumstances.
Also, the Jetty HTTP client which is used by the Solr client has an internal
feature which explicitly acquires direct ByteBuffers (see
org.eclipse.jetty.io.ArrayRetainableByteBufferPool), which could very well
explain the observed behaviour.
> Direct buffer memory leak in Solr Java Client
> ---------------------------------------------
>
> Key: SOLR-17376
> URL: https://issues.apache.org/jira/browse/SOLR-17376
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: clients - java, http2
> Affects Versions: 9.6.1
> Environment: Java version: 21.0.3+9-LTS
> Windows Server 2019
>
> Reporter: Alexander Veit
> Priority: Major
> Labels: memory-bug
> Attachments: Bildschirmfoto vom 2024-07-18 13-23-06.png,
> image-2024-07-23-14-38-17-254.png
>
>
> We use the Solr Java client in a log-running service to update documents in
> an index. As it turns out the Solr client allocates direct buffers that
> accumulate and never get released until JVM restart. After some hundreds of
> thousands instances of java.nio.DirectByteBuffer haven been created the
> system gets stuck and OutOfMemoryErrors occur inside the JVM.
> !Bildschirmfoto vom 2024-07-18 13-23-06.png!
> Unfortunately the JVM does not terminate itself, probably because this is not
> an issue of Java heap memory. Therefore the service could not be restarted
> automatically but requires user interaction.
> Although we have a rather complex scenario, we can almost certainly rule out
> that the leak is due to missing close() calls on the SolrClient instances
> within the JVM. SolrClients are generally reused during the lifetime of the
> JVM.
> The Solr client was created like in the following code
> {code:java}
> SolrClient client = new Http2SolrClient.Builder("my-url").build();{code}
>
> The Solr requests were made like in the following code
> {code:java}
> QueryRequest request = new QueryRequest(params);
>
> request.process(client, "my-collection");{code}
> A heap dump taken from an affected JVM showed the following instance counts:
> {noformat}
> org.apache.solr.client.solrj.impl.Http2SolrClient 2
> org.eclipse.jetty.io.LogarithmicArrayByteBufferPool 1
> org.eclipse.jetty.io.MappedByteBufferPool 2
> java.nio.DirectByteBuffer 458485{noformat}
>
> The DirectByteBuffers seem to be held alive by a thread-local map.
> !image-2024-07-23-14-38-17-254.png!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
