Peter Palaga created CXF-9275:
---------------------------------
Summary: HTTPConduit overwrites a decoupled SOAP reply with a late
empty HTTP 202 acknowledgement
Key: CXF-9275
URL: https://issues.apache.org/jira/browse/CXF-9275
Project: CXF
Issue Type: Bug
Reporter: Peter Palaga
We are hitting this issue once every few weeks in Quarkus CXF nightly testing.
Last time it happened today in workflow run [43786714215, job
133035888853](https://github.com/quarkiverse/quarkus-cxf/actions/runs/43786714215/job/133035888853).
The failure is in the native `WsReliableMessagingIT.testTwowayMessageLoss`
test, that failed on `hello("Paul")`
with `URLConnectionHTTPConduit` selected.
The scenario covers a two-way request with a decoupled response endpoint.
Here is an excerpt from the log:
```text
04:10:35,904 ... (executor-thread-2) RESP_IN
ExchangeId: 90256eb1-b505-4ec3-9ca0-f6defbcc9c4c
...
<return>WS-ReliableMessaging Hello Paul! counter: 3</return>
...
04:10:35,906 ... (executor-thread-1) ERROR
[io.quarkus.vertx.http.runtime.QuarkusErrorHandler] (executor-thread-1) HTTP
Request to /wsrm-rest/hello failed, error id:
b2c98abd-64c6-4c3f-ad9d-5f7678e54654-1: jakarta.xml.ws.soap.SOAPFaultException:
Response message did not contain proper response data. Expected:
{https://quarkiverse.github.io/quarkiverse-docs/quarkus-cxf/test/ws-rm}helloResponse
at
org.apache.cxf.jaxws.JaxWsClientProxy.mapException(JaxWsClientProxy.java:195)
at
org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
at
io.quarkiverse.cxf.QuarkusJaxWsProxyFactoryBean$QuarkusJaxWsClientProxy.invoke(QuarkusJaxWsProxyFactoryBean.java:171)
at jdk.proxy4/jdk.proxy4.$Proxy/s8c77c3a5.hello(Unknown Source)
at
io.quarkiverse.cxf.it.ws.rm.client.WsrmResource.hello(WsrmResource.java:38)
at
io.quarkiverse.cxf.it.ws.rm.client.WsrmResource$quarkusrestinvoker$hello_e685de476f86645c1abea6f0792a7e053a5217b1.invoke(Unknown
Source)
at
org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
at
io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:195)
at
org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:150)
at
io.quarkus.vertx.core.runtime.VertxCoreRecorder$15.runWith(VertxCoreRecorder.java:695)
at
org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2651)
at
org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2630)
at
org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1622)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1589)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at
org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
at
org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:832)
at
org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:808)
Caused by: org.apache.cxf.endpoint.ClientImpl$IllegalEmptyResponseException:
Response message did not contain proper response data. Expected:
{https://quarkiverse.github.io/quarkiverse-docs/quarkus-cxf/test/ws-rm}helloResponse
at org.apache.cxf.endpoint.ClientImpl.processResult(ClientImpl.java:675)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:539)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at
org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
... 19 more
Request method: POST
Request URI: http://localhost:8081/wsrm-rest/hello
Proxy: <none>
Request params: <none>
Query params: <none>
Form params: <none>
Path params: <none>
Headers: Accept=*/*
Content-Type=text/plain; charset=ISO-8859-1
Cookies: <none>
Multiparts: <none>
Body:
Paul
HTTP/1.1 500 Internal Server Error
content-type: application/json; charset=utf-8
content-length: 72
{
"details": "Error id b2c98abd-64c6-4c3f-ad9d-5f7678e54654-1",
"stack": ""
}
java.io.IOException: Stream closed
at
java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:267)
at
io.quarkiverse.cxf.it.ws.rm.client.WsrmServer.lambda$new$0(WsrmServer.java:63)
at java.base/java.lang.Thread.run(Thread.java:840)
```
I have let Codex analyze what happens. It said that, the outcome of the
scenario can depend on the relative order
in which the SOAP reply and the empty HTTP 202 acknowledgement are processed.
If the decoded SOAP reply has been published to the exchange, a subsequently
processed acknowledgement
can replace it before the calling thread reads the result.
The incoming observer can therefore receive and decode the correct body, yet
the synchronous call
throws `ClientImpl$IllegalEmptyResponseException` when it reads the overwritten
exchange.
The inbound log records a SOAP response containing the expected result,
followed two milliseconds later
by a missing-response-data error on the calling thread.
This establishes that the response reached CXF's inbound logging interceptor;
the log alone does not establish when unmarshalling finished or when the HTTP
202 was processed.
Codex further claimed that the expected behavior is that the empty transport
acknowledgement never replaces the decoupled application response
or marks its inbound chain complete, regardless of delivery order.
Here is the a reproducing test that Codex generated:
https://github.com/apache/cxf/pull/3723/changes/cd2fd69fced1e7d70813d431f303d2e9759b2910
and the fix it has proposed
https://github.com/apache/cxf/pull/3723/changes/b9cefcf6aef191441167ce8de9d4443eae18d9b8
I do not understand the internals of this scenario good enough to be able to
tell whether the analysis is correct or whether the fix is right.
Could please somebody review this and decide what should happen next?
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)