[ https://issues.apache.org/jira/browse/SOLR-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868082#comment-17868082 ]
Michael Gibney commented on SOLR-17376: --------------------------------------- Depending on your usage pattern, [this post|https://www.evanjones.ca/java-bytebuffer-leak.html] might be relevant? If so, the sysprop {{-Djdk.nio.maxCachedBufferSize}} may help, and/or adjusting workloads to not write large heap ByteBuffers all in one shot? > Direct buffer memory leak in Solr Java Client > --------------------------------------------- > > Key: SOLR-17376 > URL: https://issues.apache.org/jira/browse/SOLR-17376 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, http2 > Affects Versions: 9.6.1 > Environment: Java version: 21.0.3+9-LTS > Windows Server 2019 > > Reporter: Alexander Veit > Priority: Major > Labels: memory-bug > Attachments: Bildschirmfoto vom 2024-07-18 13-23-06.png, > image-2024-07-23-14-38-17-254.png > > > We use the Solr Java client in a log-running service to update documents in > an index. As it turns out the Solr client allocates direct buffers that > accumulate and never get released until JVM restart. After some hundreds of > thousands instances of java.nio.DirectByteBuffer haven been created the > system gets stuck and OutOfMemoryErrors occur inside the JVM. > !Bildschirmfoto vom 2024-07-18 13-23-06.png! > Unfortunately the JVM does not terminate itself, probably because this is not > an issue of Java heap memory. Therefore the service could not be restarted > automatically but requires user interaction. > Although we have a rather complex scenario, we can almost certainly rule out > that the leak is due to missing close() calls on the SolrClient instances > within the JVM. SolrClients are generally reused during the lifetime of the > JVM. > The Solr client was created like in the following code > {code:java} > SolrClient client = new Http2SolrClient.Builder("my-url").build();{code} > > The Solr requests were made like in the following code > {code:java} > QueryRequest request = new QueryRequest(params); > > request.process(client, "my-collection");{code} > A heap dump taken from an affected JVM showed the following instance counts: > {noformat} > org.apache.solr.client.solrj.impl.Http2SolrClient 2 > org.eclipse.jetty.io.LogarithmicArrayByteBufferPool 1 > org.eclipse.jetty.io.MappedByteBufferPool 2 > java.nio.DirectByteBuffer 458485{noformat} > > The DirectByteBuffers seem to be held alive by a thread-local map. > !image-2024-07-23-14-38-17-254.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org