[
https://issues.apache.org/jira/browse/HTTPCORE-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saksham updated HTTPCORE-675:
-----------------------------
Description:
We are using *CloseableHttpClient* with *PoolingHttpClientConnectionManager,*
following is the configuration:
{code:java}
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(1000)
.setConnectionRequestTimeout(1000)
.setSocketTimeout(1000)
.build();
PoolingHttpClientConnectionManager connectionManager = new
PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(100);
connectionManager.setDefaultMaxPerRoute(100);
this.httpClient = HttpClients.custom()
.setDefaultRequestConfig(requestConfig)
.setConnectionManager(connectionManager)
.build();
{code}
Now when we are not consuming the response entity and just relying on
closeableHttpResponse to release the resources, we end up with 25K connections
in CLOSE_WAIT state with just over 400RPS to the system. Due this system runs
out of resources very quickly and requests start failing.
Problem Snippet:
{code:java}
try (CloseableHttpResponse response = this.httpClient.execute(httpGet)){
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
return EntityUtils.toByteArray(response.getEntity());
} else {
if (response.getStatusLine().getStatusCode() >= 500) {
throw new RemoteCallFailException();
}
}
} catch (IOException e) {
logger.error("Remote db read request failed [{}]", e.getMessage(), e);
throw new RemoteCallFailException();
}
{code}
After using *EntityUtils.toByteArray(response.getEntity()) in the else block*
of above code connections are properly getting released and reused in
subsequent requests.
Working Snippet:
{code:java}
try (CloseableHttpResponse response = this.httpClient.execute(httpGet)) {
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
return EntityUtils.toByteArray(response.getEntity());
} else {
// Consuming stream to release connection
EntityUtils.toByteArray(response.getEntity());
if (response.getStatusLine().getStatusCode() >= 500) {
throw new RemoteCallFailException();
}
}
} catch (IOException e) {
logger.error("Remote db read request failed [{}]", e.getMessage(), e);
throw new RemoteCallFailException();
}
{code}
Can someone please point why releasing of connection is not automatically done
by closeablehttpResponse.close() ? ( Or maybe point to something wrong in the
code)
MVN dependency used for same:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.3</version>
</dependency>
was:
We are using *CloseableHttpClient* with *PoolingHttpClientConnectionManager,*
following is the configuration:
{code:java}
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(1000)
.setConnectionRequestTimeout(1000)
.setSocketTimeout(1000)
.build();
PoolingHttpClientConnectionManager connectionManager = new
PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(100);
connectionManager.setDefaultMaxPerRoute(100);
this.httpClient = HttpClients.custom()
.setDefaultRequestConfig(requestConfig)
.setConnectionManager(connectionManager)
.build();
{code}
Now when we are not consuming the response entity and just relying on
closeableHttpResponse to release the resources, we end up with 25K connections
in CLOSE_WAIT state with just over 400RPS to the system. Due this system runs
out of resources very quickly and requests start failing.
Problem Snippet:
{code:java}
try (CloseableHttpResponse response = this.httpClient.execute(httpGet)){
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
return EntityUtils.toByteArray(response.getEntity());
} else {
if (response.getStatusLine().getStatusCode() >= 500) {
throw new RemoteCallFailException();
}
}
} catch (IOException e) {
logger.error("Remote db read request failed [{}]", e.getMessage(), e);
throw new RemoteCallFailException();
}
{code}
After using *EntityUtils.consume(response.getEntity()) in the else block* of
above code connections are properly getting released and reused in subsequent
requests.
Working Snippet:
{code:java}
try (CloseableHttpResponse response = this.httpClient.execute(httpGet)) {
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
return EntityUtils.toByteArray(response.getEntity());
} else {
// Consuming stream to release connection
EntityUtils.toByteArray(response.getEntity());
if (response.getStatusLine().getStatusCode() >= 500) {
throw new RemoteCallFailException();
}
}
} catch (IOException e) {
logger.error("Remote db read request failed [{}]", e.getMessage(), e);
throw new RemoteCallFailException();
}
{code}
Can someone please point why releasing of connection is not automatically done
by closeablehttpResponse.close() ? ( Or maybe point to something wrong in the
code)
MVN dependency used for same:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.3</version>
</dependency>
> CloseableHttpClient leaving too many TIME_WAIT connections
> -----------------------------------------------------------
>
> Key: HTTPCORE-675
> URL: https://issues.apache.org/jira/browse/HTTPCORE-675
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Reporter: Saksham
> Priority: Major
>
> We are using *CloseableHttpClient* with *PoolingHttpClientConnectionManager,*
> following is the configuration:
> {code:java}
> RequestConfig requestConfig = RequestConfig.custom()
> .setConnectTimeout(1000)
> .setConnectionRequestTimeout(1000)
> .setSocketTimeout(1000)
> .build();
> PoolingHttpClientConnectionManager connectionManager = new
> PoolingHttpClientConnectionManager();
> connectionManager.setMaxTotal(100);
> connectionManager.setDefaultMaxPerRoute(100);
> this.httpClient = HttpClients.custom()
> .setDefaultRequestConfig(requestConfig)
> .setConnectionManager(connectionManager)
> .build();
> {code}
>
> Now when we are not consuming the response entity and just relying on
> closeableHttpResponse to release the resources, we end up with 25K
> connections in CLOSE_WAIT state with just over 400RPS to the system. Due this
> system runs out of resources very quickly and requests start failing.
> Problem Snippet:
> {code:java}
> try (CloseableHttpResponse response = this.httpClient.execute(httpGet)){
> if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
> return EntityUtils.toByteArray(response.getEntity());
> } else {
> if (response.getStatusLine().getStatusCode() >= 500) {
> throw new RemoteCallFailException();
> }
> }
> } catch (IOException e) {
> logger.error("Remote db read request failed [{}]", e.getMessage(), e);
> throw new RemoteCallFailException();
> }
> {code}
> After using *EntityUtils.toByteArray(response.getEntity()) in the else
> block* of above code connections are properly getting released and reused in
> subsequent requests.
> Working Snippet:
> {code:java}
> try (CloseableHttpResponse response = this.httpClient.execute(httpGet)) {
> if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
> return EntityUtils.toByteArray(response.getEntity());
> } else {
> // Consuming stream to release connection
> EntityUtils.toByteArray(response.getEntity());
> if (response.getStatusLine().getStatusCode() >= 500) {
> throw new RemoteCallFailException();
> }
> }
> } catch (IOException e) {
> logger.error("Remote db read request failed [{}]", e.getMessage(), e);
> throw new RemoteCallFailException();
> }
> {code}
> Can someone please point why releasing of connection is not automatically
> done by closeablehttpResponse.close() ? ( Or maybe point to something wrong
> in the code)
> MVN dependency used for same:
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.3</version>
> </dependency>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
