Hi Ryan I am not sure understand what exactly you are trying to do here? Do you intentionally use a deprecated method _and_ fail to correctly return connections back to the pool? Is this the scenario you are trying to test?
Oleg On Tue, 2025-06-03 at 14:15 -0700, Ryan Schmitt wrote: > Furthermore, I can confirm that reverting ee0a10210 fixes the issue. > No httpcore5 changes are required. > > On Tue, Jun 3, 2025 at 2:10 PM Ryan Schmitt <rschm...@apache.org> > wrote: > > > > Here's a reproducer: > > > > https://github.com/rschmitt/httpcomponents-client/tree/gc-repro > > > > The bug is exhibited on JDK11 and JDK21, not on JDK8 or JDK21. I > > didn't test with any non-LTS releases. Here's JDK17: > > > > [0] ~/src/httpcomponents-client # j17 ./run-example.sh > > SocketLeak > > This Java version SHOULD exhibit the bug where two garbage > > collections are required to clean up leaked sockets. > > Leaking client; sockets should be in ESTABLISHED > > Tue Jun 3 14:04:40 PDT 2025: > > Tue Jun 3 14:04:41 PDT 2025: 2 (ESTABLISHED) > > Tue Jun 3 14:04:42 PDT 2025: 3 (ESTABLISHED) > > Tue Jun 3 14:04:44 PDT 2025: 3 (ESTABLISHED) > > Running garbage collector; sockets should be in CLOSE_WAIT > > Tue Jun 3 14:04:45 PDT 2025: 3 (CLOSE_WAIT) > > Tue Jun 3 14:04:46 PDT 2025: 3 (CLOSE_WAIT) > > Tue Jun 3 14:04:47 PDT 2025: 3 (CLOSE_WAIT) > > Running garbage collector again; no sockets should be listed > > Tue Jun 3 14:04:48 PDT 2025: > > Tue Jun 3 14:04:50 PDT 2025: > > > > And here's JDK21: > > > > [0] ~/src/httpcomponents-client # j21 ./run-example.sh > > SocketLeak > > This Java version SHOULD NOT exhibit the bug where two garbage > > collections are required to clean up leaked sockets. > > Leaking client; sockets should be in ESTABLISHED > > Tue Jun 3 14:06:25 PDT 2025: > > Tue Jun 3 14:06:26 PDT 2025: 3 (ESTABLISHED) > > Tue Jun 3 14:06:28 PDT 2025: 3 (ESTABLISHED) > > Tue Jun 3 14:06:29 PDT 2025: 3 (ESTABLISHED) > > Running garbage collector; no sockets should be listed > > Tue Jun 3 14:06:30 PDT 2025: > > Tue Jun 3 14:06:31 PDT 2025: > > > > I tested this on both macOS and Linux. > > > > On Fri, May 30, 2025 at 5:14 AM Oleg Kalnichevski > > <ol...@apache.org> wrote: > > > > > > On Fri, 2025-05-30 at 11:30 +0200, Oleg Kalnichevski wrote: > > > > On Thu, 2025-05-29 at 22:54 -0700, Ryan Schmitt wrote: > > > > > It's the `autoClose` setting. It used to be `true`: > > > > > > > > > > https://github.com/apache/httpcomponents-client/blob/d8f702fb4d44c746bb0edf00643aa7139cb8bdf7/httpclient5/src/main/java/org/apache/hc/client5/http/ssl/SSLConnectionSocketFactory.java#L274 > > > > > > > > > > Now it's `false`: > > > > > > > > > > https://github.com/apache/httpcomponents-client/blob/ffe4a05f4faf3706a41ff660bcb3474c6b5101a3/httpclient5/src/main/java/org/apache/hc/client5/http/ssl/AbstractClientTlsStrategy.java#L208 > > > > > > > > > > This was deliberately changed in this commit: > > > > > > > > > > https://github.com/apache/httpcomponents-client/commit/ee0a102104d03a8d1cfe18e571d179c41242182c > > > > > > > > > > > > > This change was introduced in response to HTTPCLIENT-2328 [1]. > > > > I will > > > > review the code for accidental mistakes. Conceptually the > > > > `autoClose` > > > > setting should be merely a convenience. > > > > > > > > Oleg > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/HTTPCLIENT-2328 > > > > > > > > > > I think this change in core is more relevant [1]. At the moment I > > > cannot see anything obviously wrong with it. > > > > > > Would you be able to share (contribute) a reproducer for the > > > defect? > > > > > > Oleg > > > > > > [1] > > > https://github.com/apache/httpcomponents-core/commit/1c95b2994f22b91bc7f54cb66ed84a8e94111a61 > > > > > > > > > > > > > > > > On Thu, May 29, 2025 at 6:08 PM Ryan Schmitt > > > > > <rschm...@apache.org> > > > > > wrote: > > > > > > > > > > > > I've been debugging a load test regression, which turned > > > > > > out to > > > > > > be > > > > > > caused by a client instance leak. The test started failing > > > > > > on > > > > > > 5.4.4 > > > > > > due to running out of file descriptors. What I realized is > > > > > > that > > > > > > leaked > > > > > > connection pools are eventually cleaned up by garbage > > > > > > collection, > > > > > > which causes the sockets' file descriptors to be released. > > > > > > What > > > > > > has > > > > > > changed in 5.4.4 is that this process now takes TWO rounds > > > > > > of > > > > > > garbage > > > > > > collection. This can be seen in the `lsof` output, which > > > > > > shows > > > > > > where > > > > > > in the TCP state machine the socket is: > > > > > > > > > > > > 5.2: ESTABLISHED -(gc)-> (gone) > > > > > > 5.4: ESTABLISHED -(gc)-> CLOSE_WAIT -(gc)-> (gone) > > > > > > > > > > > > Does anyone know what might have changed that would cause > > > > > > this? > > > > > > I'm > > > > > > specifically asking about the synchronous client (I haven't > > > > > > tested > > > > > > the > > > > > > async client). > > > > > > > > > > ------------------------------------------------------------- > > > > > ------ > > > > > -- > > > > > To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org > > > > > For additional commands, e-mail: dev-h...@hc.apache.org > > > > > > > > > > > > > > > > > --------------------------------------------------------------- > > > > ------ > > > > To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org > > > > For additional commands, e-mail: dev-h...@hc.apache.org > > > > > > > > > > > > > ----------------------------------------------------------------- > > > ---- > > > To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org > > > For additional commands, e-mail: dev-h...@hc.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org > For additional commands, e-mail: dev-h...@hc.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
