[ 
https://issues.apache.org/jira/browse/SOLR-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868082#comment-17868082
 ] 

Michael Gibney commented on SOLR-17376:
---------------------------------------

Depending on your usage pattern, [this 
post|https://www.evanjones.ca/java-bytebuffer-leak.html] might be relevant? If 
so, the sysprop {{-Djdk.nio.maxCachedBufferSize}} may help, and/or adjusting 
workloads to not write large heap ByteBuffers all in one shot?

> Direct buffer memory leak in Solr Java Client
> ---------------------------------------------
>
>                 Key: SOLR-17376
>                 URL: https://issues.apache.org/jira/browse/SOLR-17376
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: clients - java, http2
>    Affects Versions: 9.6.1
>         Environment: Java version: 21.0.3+9-LTS
> Windows Server 2019
>  
>            Reporter: Alexander Veit
>            Priority: Major
>              Labels: memory-bug
>         Attachments: Bildschirmfoto vom 2024-07-18 13-23-06.png, 
> image-2024-07-23-14-38-17-254.png
>
>
> We use the Solr Java client in a log-running service to update documents in 
> an index. As it turns out the Solr client allocates direct buffers that 
> accumulate and never get released until JVM restart. After some hundreds of 
> thousands instances of java.nio.DirectByteBuffer haven been created the 
> system gets stuck and OutOfMemoryErrors occur inside the JVM.
> !Bildschirmfoto vom 2024-07-18 13-23-06.png!
> Unfortunately the JVM does not terminate itself, probably because this is not 
> an issue of Java heap memory. Therefore the service could not be restarted 
> automatically but requires user interaction.
> Although we have a rather complex scenario, we can almost certainly rule out 
> that the leak is due to missing close() calls on the SolrClient instances 
> within the JVM. SolrClients are generally reused during the lifetime of the 
> JVM.
> The Solr client was created like in the following code
> {code:java}
> SolrClient client = new Http2SolrClient.Builder("my-url").build();{code}
>  
> The Solr requests were made like in the following code
> {code:java}
> QueryRequest request = new QueryRequest(params);
>  
> request.process(client, "my-collection");{code}
> A heap dump taken from an affected JVM showed the following instance counts:
> {noformat}
> org.apache.solr.client.solrj.impl.Http2SolrClient        2
> org.eclipse.jetty.io.LogarithmicArrayByteBufferPool      1
> org.eclipse.jetty.io.MappedByteBufferPool                2
> java.nio.DirectByteBuffer                           458485{noformat}
>  
> The DirectByteBuffers seem to be held alive by a thread-local map.
> !image-2024-07-23-14-38-17-254.png!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org

Reply via email to