[
https://issues.apache.org/jira/browse/CXF-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755370#comment-17755370
]
Jim Ma commented on CXF-8903:
-----------------------------
To reproduce this issue easily, I created branch :
[https://github.com/jimma/cxf/tree/too-many-open-files-issue.]
Build the testutils module first, then run {_}"mvn clean install -mvn clean
install -Dtest=ClientServerMiscTest#testHelloWSTimes{_}". This new added test
creates bunch of jaxws client and print the total open files descriptor count
every 200ms.
>From my local run, this test opens about 2700 files for the 500 clients
>invocation,
{code:java}
[INFO] Running org.apache.cxf.systest.jaxws.ClientServerMiscTest
CatalogManager.properties: catalogs not found.
CatalogManager.properties: catalogs not found.
* Before test the opened files count :173
opened files count :175
opened files count :225
opened files count :225
opened files count :568
opened files count :830
opened files count :1125
opened files count :1483
opened files count :1834
opened files count :2125
opened files count :2513
opened files count :2899
opened files count :3043
opened files count :1524
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.522 s
-- in org.apache.cxf.systest.jaxws.ClientServerMiscTest{code}
but the old HttpURLConnection in CXF
4.0.0(https://github.com/jimma/cxf/tree/4.0.0-openfiles) only opens about 20
files.
{code:java}
[INFO] Running org.apache.cxf.systest.jaxws.ClientServerMiscTest
* Before test the opened files count :167
opened files count :187
opened files count :187
opened files count :179
opened files count :179
opened files count :186
opened files count :183
opened files count :161
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.923 s
- in org.apache.cxf.systest.jaxws.ClientServerMiscTest {code}
I use lsof to look at the opened files by HttpClient and it shows the
connection is still there without being closed after the
:[https://gist.github.com/jimma/ea0cb67465229e67a2c8dcd231a89d7f.]
I didn't see this happens for the old URLConnectionConduit.
I tried to switch to 22ea JDK and add the HttpClient.close() when the
HttpConduit is closed, but this doesn't close the connection and open files are
still the same.
> Too many open files error in HttpClientHTTPConduit
> --------------------------------------------------
>
> Key: CXF-8903
> URL: https://issues.apache.org/jira/browse/CXF-8903
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 4.0.2
> Reporter: Jim Ma
> Priority: Major
> Fix For: 4.0.3, 4.1.0
>
>
> It seems there is resource leak in HttpClientHTTPConduit. When upgrade CXF to
> 4.0.2, the "lsof" shows there are many files open since it is changed to use
> JDK's HTTPClient.
> {code:java}
> Caused by: java.lang.InternalError: java.io.IOException: Too many open files
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.<init>(HttpClientImpl.java:311)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.create(HttpClientImpl.java:253)
> at
> java.net.http/jdk.internal.net.http.HttpClientBuilderImpl.build(HttpClientBuilderImpl.java:135)
> at
> org.apache.cxf.impl//org.apache.cxf.transport.http.HttpClientHTTPConduit.setupConnection(HttpClientHTTPConduit.java:231)
> at
> org.apache.cxf.impl//org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduit.setupConnection(AsyncHTTPConduit.java:190)
> at
> org.apache.cxf.impl//org.apache.cxf.transport.http.HTTPConduit.prepare(HTTPConduit.java:529)
> at
> org.apache.cxf@4.0.2//org.apache.cxf.transport.TransportURIResolver.resolve(TransportURIResolver.java:133)
> at
> org.apache.cxf@4.0.2//org.apache.cxf.catalog.CatalogXmlSchemaURIResolver.resolveEntity(CatalogXmlSchemaURIResolver.java:66)
> at
> org.apache.ws.xmlschema//org.apache.ws.commons.schema.SchemaBuilder.resolveXmlSchema(SchemaBuilder.java:682)
> at
> org.apache.ws.xmlschema//org.apache.ws.commons.schema.SchemaBuilder.handleImport(SchemaBuilder.java:536)
> at
> org.apache.ws.xmlschema//org.apache.ws.commons.schema.SchemaBuilder.handleSchemaElementChild(SchemaBuilder.java:1524)
> at
> org.apache.ws.xmlschema//org.apache.ws.commons.schema.SchemaBuilder.handleXmlSchemaElement(SchemaBuilder.java:656)
> at
> org.apache.ws.xmlschema//org.apache.ws.commons.schema.XmlSchemaCollection.read(XmlSchemaCollection.java:551)
> at
> org.apache.cxf@4.0.2//org.apache.cxf.common.xmlschema.SchemaCollection.read(SchemaCollection.java:129)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.SchemaUtil.extractSchema(SchemaUtil.java:145)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.SchemaUtil.getSchemas(SchemaUtil.java:74)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.SchemaUtil.getSchemas(SchemaUtil.java:66)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.SchemaUtil.getSchemas(SchemaUtil.java:61)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.WSDLServiceBuilder.getSchemas(WSDLServiceBuilder.java:378)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.WSDLServiceBuilder.buildServices(WSDLServiceBuilder.java:345)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.WSDLServiceBuilder.buildServices(WSDLServiceBuilder.java:209)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl11.WSDLServiceFactory.create(WSDLServiceFactory.java:161)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:407)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:527)
> at
> org.apache.cxf.impl//org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:262)
> at
> org.apache.cxf.impl//org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:199)
> at
> org.apache.cxf.impl//org.apache.cxf.jaxws.ServiceImpl.createPort(ServiceImpl.java:466)
> at
> org.jboss.ws.jaxws-client@7.0.0-SNAPSHOT//org.jboss.wsf.stack.cxf.client.ProviderImpl$JBossWSServiceImpl.createPort(ProviderImpl.java:580)
> at
> org.apache.cxf.impl//org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:342)
> at
> org.apache.cxf.impl//org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:337)
> at
> jakarta.xml.ws.api@1.0.0.Final//jakarta.xml.ws.Service.getPort(Service.java:210)
> at
> deployment.jaxws-cxf-jbws3713-client.war//org.jboss.test.ws.jaxws.cxf.jbws3713.HelperUsignThreadLocal$3.initialValue(HelperUsignThreadLocal.java:145)
> at
> deployment.jaxws-cxf-jbws3713-client.war//org.jboss.test.ws.jaxws.cxf.jbws3713.HelperUsignThreadLocal$3.initialValue(HelperUsignThreadLocal.java:139)
> at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
> at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
> at
> deployment.jaxws-cxf-jbws3713-client.war//org.jboss.test.ws.jaxws.cxf.jbws3713.HelperUsignThreadLocal$Client.call(HelperUsignThreadLocal.java:167)
> at
> deployment.jaxws-cxf-jbws3713-client.war//org.jboss.test.ws.jaxws.cxf.jbws3713.HelperUsignThreadLocal$Client.call(HelperUsignThreadLocal.java:153)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ... 1 more
> Caused by: java.io.IOException: Too many open files
> at java.base/sun.nio.ch.KQueue.create(Native Method)
> at
> java.base/sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:82)
> at
> java.base/sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:35)
> at java.base/java.nio.channels.Selector.open(Selector.java:295)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.<init>(HttpClientImpl.java:699)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.<init>(HttpClientImpl.java:308)
> ... 40 more{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
