David Richter created CXF-9234:
----------------------------------
Summary: Concurrent DynamicClientFactory.createClient for the same
WSDL url corrupts the cached schema DOM and spins forever at 100% CPU
Key: CXF-9234
URL: https://issues.apache.org/jira/browse/CXF-9234
Project: CXF
Issue Type: Bug
Components: Simple Frontend
Affects Versions: 4.2.2
Environment: CXF 4.2.2, Woodstox 7.2.0
xerces:xercesImpl:2.12.2
Java Azul 25.0.3
Arch Linux, also reproduced on Windows 11
Reporter: David Richter
Attachments: cxf-concurrent-schema-clone-repro.zip
Two threads calling `DynamicClientFactory.createClient(wsdlUrl, ...)` for the
*same* url can enter an unbounded spin at 100% CPU and never return. The thread
is RUNNABLE, ignores interrupts, and keeps burning a core until the JVM exits.
h2. Stacktrace:
{code:java}
java.lang.Thread.State: RUNNABLE
at java.util.WeakHashMap.get([email protected]/WeakHashMap.java:416)
at org.apache.xerces.dom.CoreDocumentImpl.setUserData(Unknown Source)
at org.apache.xerces.dom.NodeImpl.setUserData(Unknown Source)
at
org.apache.cxf.staxutils.StaxUtils$LocationUserDataHandler.handle(StaxUtils.java:1648)
at org.apache.xerces.dom.CoreDocumentImpl.callUserDataHandlers(Unknown
Source)
at org.apache.xerces.dom.CoreDocumentImpl.callUserDataHandlers(Unknown
Source)
at org.apache.xerces.dom.NodeImpl.cloneNode(Unknown Source)
at org.apache.xerces.dom.ChildNode.cloneNode(Unknown Source)
at org.apache.xerces.dom.ParentNode.cloneNode(Unknown Source)
at org.apache.xerces.dom.ElementImpl.cloneNode(Unknown Source)
at org.apache.xerces.dom.ParentNode.cloneNode(Unknown Source)
at org.apache.xerces.dom.ElementImpl.cloneNode(Unknown Source)
at org.apache.xerces.dom.ParentNode.cloneNode(Unknown Source)
at org.apache.xerces.dom.ElementImpl.cloneNode(Unknown Source)
at org.apache.xerces.dom.ParentNode.cloneNode(Unknown Source)
at org.apache.xerces.dom.ElementImpl.cloneNode(Unknown Source)
at
org.apache.cxf.endpoint.dynamic.DynamicClientFactory.cloneNode(DynamicClientFactory.java:986)
at
org.apache.cxf.endpoint.dynamic.DynamicClientFactory.removeImportElement(DynamicClientFactory.java:946)
at
org.apache.cxf.endpoint.dynamic.DynamicClientFactory.addSchemas(DynamicClientFactory.java:593)
at
org.apache.cxf.endpoint.dynamic.DynamicClientFactory.createClient(DynamicClientFactory.java:337)
at
org.apache.cxf.endpoint.dynamic.DynamicClientFactory.createClient(DynamicClientFactory.java:225)
...{code}
h2. Cause (AI generated during investigation but seems legit):
1. `WSDLManagerImpl` caches the parsed `Definition` — and its schema DOM — per
WSDL url. Map access is synchronized, but the cached DOM is handed to
concurrent callers unsynchronized.
2. `DynamicClientFactory.removeImportElement` deep-clones that schema when it
contains an `xs:import`/`xs:include` (`DynamicClientFactory:946`; `owner ==
document`, so this delegates to `node.cloneNode(deep)` on the cached document).
3. Every cloned node re-fires `StaxUtils$LocationUserDataHandler.handle` →
`dst.setUserData(...)` (`StaxUtils:1643-1650`), writing into the **source**
document's userData map.
4. That map is not thread safe in any DOM implementation — `WeakHashMap` in
Xerces-J, `HashMap` in the JDK DOM — so concurrent clones corrupt it into a
cycle and the lookups then loop forever.
h2. Reproducer project (AI generated manually checked)
`cxf-concurrent-schema-clone-repro.zip` — self-contained Maven project, Maven
wrapper included, no network service needed.
{code:java}
./mvnw test # JDK built-in DOM
./mvnw test -Pxerces # Xerces-J 2.12.2
{code}
Both fail on `concurrentDeepCloneOfSharedSchemaMustNotHang`, printing the
worker stacks to stderr. Select a CXF version with `-Dcxf.version=4.1.4`.
The project has three tests: the failing one deep-clones a single shared
location-tracked schema from N threads (what step 2 above does); a
single-threaded control makes the same number of clones in ~4 s, showing the
problem is the concurrency and not the schema size; and one exercises the real
`createClient` path with two threads on a shared bus — that one is timing
dependent and passed on an idle machine here.
It is a race, so the failing test retries fresh rounds for up to 180 s before
giving up (it tripped on round 1–3 on a 22-core box).
`observed-thread-dumps.txt` in the archive holds real dumps from both DOM
variants in case it does not trip on your hardware.
Note: once it trips, the worker threads cannot be stopped — expect to kill the
JVM.
h2. Workaround
Build each client on its own Bus, so the per-bus WSDLManager cache - and
therefore the
schema DOM - is not shared between threads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
