[
https://issues.apache.org/jira/browse/CXF-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18141958#comment-18141958
]
Andriy Redko commented on CXF-9272:
-----------------------------------
Sorry [~ffang] , I assigned to myself by mistake
> HttpClientHTTPConduit: async call to a refused connection sometimes never
> calls the ClientCallback (regression from CXF-9250)
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: CXF-9272
> URL: https://issues.apache.org/jira/browse/CXF-9272
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 4.2.4
> Reporter: Federico Mariani
> Assignee: Freeman Yue Fang
> Priority: Major
> Fix For: 3.6.14, 4.2.5, 4.1.10
>
>
> Since 4.2.4, an asynchronous invocation through {{HttpClientHTTPConduit}} to
> an address that refuses the connection sometimes never completes. Neither
> {{ClientCallback.handleResponse}} nor {{handleException}} is called, so a
> caller waiting on the callback (or on the future) waits forever. The receive
> timeout does not help, because no response handling is ever scheduled.
> We found this through Apache Camel's camel-cxf async producer, where an
> exchange hangs forever when the target SOAP service is down (CAMEL-26696,
> CAMEL-26600). The reproducer below uses plain CXF only.
> *Reproducer* (JUnit 5, plain JAX-WS client, a fresh client per call, nothing
> listening on the port):
> {code:java}
> @WebService(targetNamespace = "http://example.org/echo")
> public interface Echo {
> String echo(String text);
> }
> @Test
> void asyncCallToClosedPortAlwaysCallsBack() throws Exception {
> int port;
> try (ServerSocket s = new ServerSocket(0)) {
> port = s.getLocalPort(); // nothing listens on it after close
> }
> String address = "http://localhost:" + port + "/echo";
> int calls = 2400;
> AtomicInteger stuck = new AtomicInteger();
> ExecutorService pool = Executors.newFixedThreadPool(16);
> CountDownLatch all = new CountDownLatch(calls);
> for (int i = 0; i < calls; i++) {
> pool.execute(() -> {
> try {
> JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
> factory.setServiceClass(Echo.class);
> factory.setAddress(address);
> Client client = ClientProxy.getClient(factory.create());
> HTTPClientPolicy policy = new HTTPClientPolicy();
> policy.setConnectionTimeout(3000);
> policy.setReceiveTimeout(3000);
> ((HTTPConduit) client.getConduit()).setClient(policy);
> CountDownLatch callback = new CountDownLatch(1);
> client.invoke(new ClientCallback() {
> @Override
> public void handleResponse(Map<String, Object> ctx,
> Object[] res) {
> super.handleResponse(ctx, res);
> callback.countDown();
> }
> @Override
> public void handleException(Map<String, Object> ctx,
> Throwable ex) {
> super.handleException(ctx, ex);
> callback.countDown();
> }
> }, "echo", "hello");
> if (!callback.await(8, TimeUnit.SECONDS)) {
> stuck.incrementAndGet();
> }
> client.destroy();
> } catch (Exception e) {
> // a synchronous failure from invoke() counts as completed
> } finally {
> all.countDown();
> }
> });
> }
> all.await();
> pool.shutdown();
> System.out.println(Version.getCurrentVersion() + ": stuck (no callback
> within 8s) = " + stuck.get());
> }
> {code}
> *Results* (2 runs of 2400 calls per version, 16 threads, JDK 25.0.2 Temurin,
> macOS aarch64):
> ||CXF||calls||no callback within 8 s||
> |4.2.3|4800|0|
> |4.2.4|4800|22 (18 + 4)|
> |4.2.5-SNAPSHOT (20261110.085644)|4800|42 (22 + 20)|
> Sequential calls rarely hit it locally (0 in 3300 calls through the Camel
> producer). Concurrency widens the race window, as does a loaded CI machine.
> *What the stuck exchanges look like*
> We instrumented the Camel-based test, which uses the same async
> {{ClientImpl.invoke(ClientCallback, ...)}} path, with an out interceptor at
> {{Phase.SETUP}} that keeps the out message. Every stuck exchange shows the
> same state:
> * {{outMessage.getContent(Exception.class)}} is
> {{java.util.concurrent.CompletionException: java.net.ConnectException}}
> * {{exchange.getInMessage()}} is null
> * {{exchange.get(ClientCallback.class)}} is still set, so {{ClientImpl}}
> never dispatched it
> The calls that do complete report a {{Fault}} caused by {{ConnectException}}
> through {{handleException}}.
> *Suspected cause* (our reading of the code; we did not instrument inside CXF)
> CXF-9250 added this to the {{future.exceptionally(...)}} callback in
> {{HttpClientHTTPConduit.HttpClientWrappedOutputStream#setProtocolHeaders}}:
> {code:java}
> outMessage.setContent(Exception.class, ex instanceof Exception ? (Exception)
> ex : new IOException(ex));
> ...
> close();
> {code}
> {{HTTPConduit.WrappedOutputStream#handleResponseOnWorkqueue}} only schedules
> the runnable that calls {{handleResponseInternal()}}, and the fault observer
> if that fails, when no exception is recorded:
> {code:java}
> boolean exceptionSet = outMessage.getContent(Exception.class) != null;
> if (!exceptionSet) {
> ... execute(runnable) ...
> }
> {code}
> If the {{exceptionally}} callback records the {{ConnectException}} before the
> caller thread's {{close()}} reaches {{handleResponse()}}, both {{close()}}
> calls (the caller's and the one from {{exceptionally}}) skip scheduling.
> Nothing then calls the fault observer, so the {{ClientCallback}} is never
> invoked. The raw {{CompletionException}} left on the out message of the stuck
> exchanges matches this path. The 4.2.3 to 4.2.4 difference also points at
> CXF-9250: between the {{cxf-4.2.3}} and {{cxf-4.2.4}} tags it is the only
> commit touching {{HttpClientHTTPConduit}} or {{HTTPConduit}}.
> CXF-9250 also went into 4.1.9, 4.0.12 and 3.6.13. We did not test those
> versions.
> Possibly related, but a different cause: CXF-9257 (the callback throws) and
> CXF-9267 (a race in {{HttpClientBodyPublisher}} hides the connect timeout).
> Both are fixed on main, and this issue still reproduces on the 4.2.5-SNAPSHOT
> above.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)