Chris,
that's good to hear, I'll try it out.
Best,
Albert
On 07.08.2018 18:53, Chris Hegarty wrote:
Albert,
I haven’t yet looked at what happens in JDK 10, but just to say, since things
have moved on a lot in JDK 11 EA, that the same test runs with a reasonable
amount of memory and CPU with JDK 11 EA [1]. I updated the test a little, since
some the names have been changed with the standardisation ( see javadoc for
more details [2] ).
-Chris.
[1] http://jdk.java.net/11/
[2]
https://download.java.net/java/early_access/jdk11/docs/api/java.net.http/java/net/http/package-summary.html
---
$ cat BadProxyLeak.java
import java.net.InetSocketAddress;
import java.net.ProxySelector;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.concurrent.TimeUnit;
public class BadProxyLeak {
public static void main(String[] args) throws InterruptedException {
// 1st heap dump
// Thread.sleep(10000);
{
HttpClient client = HttpClient.newBuilder()
// malicious proxy?
.proxy(ProxySelector.of(new
InetSocketAddress("165.165.248.90", 8080)))
.build();
HttpRequest req = HttpRequest
// target is not relevant
.newBuilder(URI.create("https://www.google.de";))
// leak occurs only with HTTP_2
.version(HttpClient.Version.HTTP_2)
// .version(HttpClient.Version.HTTP_1_1)
.GET()
.build();
// use same client for every request
// most likely happens on the first retry, if not try again or
increase i
for (int i = 0; i < 10; i++) {
System.out.println("Request " + i);
// body handler is not relevant
HttpResponse.BodyHandler<?> t =
HttpResponse.BodyHandlers.ofString();
try {
// happens with both async and sync send
client.sendAsync(req, t).get(15, TimeUnit.SECONDS);
// client.send(req, handler);
} catch (Exception e) {
e.printStackTrace();
}
}
}
// 3rd heap dump
System.out.println("Generating heap dump manually");
// space is not released
Thread.sleep(60000);
}
}
On 7 Aug 2018, at 17:05, Chris Hegarty <chris.hega...@oracle.com> wrote:
On 7 Aug 2018, at 16:55, Albert Schimpf <albi...@gmx.de> wrote:
Hi,
by bad I mean that this proxy is the only one out of ~1000 proxies which causes
this behavior. It's also the only one which causes SSLExceptions (General SSL
engine problem) and EOFExceptions. I don't think that particular proxy is a
valid proxy. Using curl indicates that this is a SSL handshake problem (https):
* Rebuilt URL to: www.google.de/
* Trying 165.165.248.90…
Oh, you mean the proxy at that actual IP address. Ok got it.
* TCP_NODELAY set
* Connected to 165.165.248.90 (165.165.248.90) port 8080 (#0)
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 165.165.248.90:8080
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to
165.165.248.90:8080
I don't know that much about the protocol itself, so I don't think I can help
very much.
Thank you for looking into this!
-Chris.
Best,
Albert
On 07.08.2018 12:05, Chris Hegarty wrote:
Hi Albert,
Very strange indeed. Thanks for reporting it, I’ll investigate.
What do your mean by “bad proxy”. What is bad about it, and how does it behave?
-Chris.
On 7 Aug 2018, at 10:25, Albert Schimpf <albi...@gmx.de> wrote:
Hi,
I stumbled upon some strange behavior when using the new Java httpclient.
The issue is very simple to reproduce. Send a GET request via a known bad proxy:
HttpClient client = HttpClient.newBuilder()
.proxy(ProxySelector.of(BAD_PROXY))
.build();
HttpRequest req = HttpRequest
// target is not relevant
.newBuilder(...)
.GET()
.build();
// body handler is not relevant
HttpResponse.BodyHandler<?> t = HttpResponse.BodyHandler.asString();
// happens with both async and sync send
client.sendAsync(req, t).get(30, TimeUnit.SECONDS);
The result is that the heap size increases dramatically (to about 1.5GB) and
resources are not released. CPU consumption increases by a constant factor,
too. I have tried many variations of the above code, and the only thing which
seems to work (i.e. heap size does not explode) is to set the HTTP version to
1.1.
In my main application this leads to both memory and CPU starvation (4GB memory
limit, 100% CPU usage). It usually uses only 5% CPU and 200MB memory at worst.
I have attached a working example code with a bad proxy. I uploaded the
generated garbage collection log and three heap dumps (before, during, and
after the request) to dropbox:
https://www.dropbox.com/s/ulqnmrmgr58rrul/debug.zip
I tried the 10.0.0-openjdk and 10.0.1-zulu version. I can reproduce the issue
100% of times.
Am I doing something wrong? Is this to be expected if one somehow happens to
use a bad proxy? If this is to be expected, how can I protect my application
against such behavior?
Best,
Albert
<mon.png><Main.java>
