Albert, I haven’t yet looked at what happens in JDK 10, but just to say, since things have moved on a lot in JDK 11 EA, that the same test runs with a reasonable amount of memory and CPU with JDK 11 EA [1]. I updated the test a little, since some the names have been changed with the standardisation ( see javadoc for more details [2] ).
-Chris. [1] http://jdk.java.net/11/ [2] https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/package-summary.html --- $ cat BadProxyLeak.java import java.net.InetSocketAddress; import java.net.ProxySelector; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.util.concurrent.TimeUnit; public class BadProxyLeak { public static void main(String[] args) throws InterruptedException { // 1st heap dump // Thread.sleep(10000); { HttpClient client = HttpClient.newBuilder() // malicious proxy? .proxy(ProxySelector.of(new InetSocketAddress("165.165.248.90", 8080))) .build(); HttpRequest req = HttpRequest // target is not relevant .newBuilder(URI.create("https://www.google.de";)) // leak occurs only with HTTP_2 .version(HttpClient.Version.HTTP_2) // .version(HttpClient.Version.HTTP_1_1) .GET() .build(); // use same client for every request // most likely happens on the first retry, if not try again or increase i for (int i = 0; i < 10; i++) { System.out.println("Request " + i); // body handler is not relevant HttpResponse.BodyHandler<?> t = HttpResponse.BodyHandlers.ofString(); try { // happens with both async and sync send client.sendAsync(req, t).get(15, TimeUnit.SECONDS); // client.send(req, handler); } catch (Exception e) { e.printStackTrace(); } } } // 3rd heap dump System.out.println("Generating heap dump manually"); // space is not released Thread.sleep(60000); } } > On 7 Aug 2018, at 17:05, Chris Hegarty <chris.hega...@oracle.com> wrote: > > >> On 7 Aug 2018, at 16:55, Albert Schimpf <albi...@gmx.de> wrote: >> >> Hi, >> >> by bad I mean that this proxy is the only one out of ~1000 proxies which >> causes this behavior. It's also the only one which causes SSLExceptions >> (General SSL engine problem) and EOFExceptions. I don't think that >> particular proxy is a valid proxy. Using curl indicates that this is a SSL >> handshake problem (https): >> >> * Rebuilt URL to: www.google.de/ >> * Trying 165.165.248.90… > > Oh, you mean the proxy at that actual IP address. Ok got it. > > >> * TCP_NODELAY set >> * Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0) >> * successfully set certificate verify locations: >> * CAfile: /etc/ssl/certs/ca-certificates.crt >> CApath: /etc/ssl/certs >> * TLSv1.2 (OUT), TLS handshake, Client hello (1): >> * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080 >> * Closing connection 0 >> curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to >> 165.165.248.90:8080 >> >> I don't know that much about the protocol itself, so I don't think I can >> help very much. >> >> Thank you for looking into this! > > > -Chris. > > >> Best, >> Albert >> >> >> On 07.08.2018 12:05, Chris Hegarty wrote: >>> Hi Albert, >>> >>> Very strange indeed. Thanks for reporting it, I’ll investigate. >>> >>> What do your mean by “bad proxy”. What is bad about it, and how does it >>> behave? >>> >>> -Chris. >>> >>> >>>> On 7 Aug 2018, at 10:25, Albert Schimpf <albi...@gmx.de> wrote: >>>> >>>> Hi, >>>> >>>> I stumbled upon some strange behavior when using the new Java httpclient. >>>> >>>> The issue is very simple to reproduce. Send a GET request via a known bad >>>> proxy: >>>> >>>> HttpClient client = HttpClient.newBuilder() >>>> .proxy(ProxySelector.of(BAD_PROXY)) >>>> .build(); >>>> >>>> HttpRequest req = HttpRequest >>>> // target is not relevant >>>> .newBuilder(...) >>>> .GET() >>>> .build(); >>>> >>>> // body handler is not relevant >>>> HttpResponse.BodyHandler<?> t = HttpResponse.BodyHandler.asString(); >>>> >>>> // happens with both async and sync send >>>> client.sendAsync(req, t).get(30, TimeUnit.SECONDS); >>>> >>>> The result is that the heap size increases dramatically (to about 1.5GB) >>>> and resources are not released. CPU consumption increases by a constant >>>> factor, too. I have tried many variations of the above code, and the only >>>> thing which seems to work (i.e. heap size does not explode) is to set the >>>> HTTP version to 1.1. >>>> >>>> In my main application this leads to both memory and CPU starvation (4GB >>>> memory limit, 100% CPU usage). It usually uses only 5% CPU and 200MB >>>> memory at worst. >>>> >>>> I have attached a working example code with a bad proxy. I uploaded the >>>> generated garbage collection log and three heap dumps (before, during, and >>>> after the request) to dropbox: >>>> >>>> https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip >>>> >>>> I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the >>>> issue 100% of times. >>>> >>>> Am I doing something wrong? Is this to be expected if one somehow happens >>>> to use a bad proxy? If this is to be expected, how can I protect my >>>> application against such behavior? >>>> >>>> >>>> Best, >>>> Albert >>>> <mon.png><Main.java> >> >
