RemiFer1 opened a new issue, #3645:
URL: https://github.com/apache/fory/issues/3645
### Search before asking
- [x] I had searched in the issues and found no open issues matching.
### Version
Fory 0.17.0 (Java)
### Component(s)
Java
### Description
We hit a deserialization failure on a deeply nested object graph where
reference tracking ends up off by exactly **one slot** the first time a
`java.util.Collections$SetFromMap` (built via `Collections.newSetFromMap(new
ConcurrentHashMap<>())`) is encountered.
Symptoms match #1832 ("SetFromMapSerializer didn't handle nested reference
right"), which was reported and closed by #1833 in Sep 2024 — so this looks
like either a regression in 0.17.0, or a slightly different call shape that PR
#1833 didn't cover.
### Minimal reproduce step
We did **not** isolate a minimal repro — the bug only surfaces deep inside a
large object graph (~76,000 ref-tracked slots before the failure point).
Instead we localized the cause empirically with paired writer/reader
instrumentation (described below).
Conditions that were present:
- `withRefTracking(true)`
- `requireClassRegistration(true)` (Fory auto-registers
`Collections$SetFromMap` at id 179)
- A `Collections.newSetFromMap(new ConcurrentHashMap<>())` instance reached
as part of a nested `Externalizable.writeExternal` -> `stream.writeObject(set)`
flow (i.e., not a top-level `Fory.serialize(set)` call).
### What did you expect to see?
Successful round-trip. Writer and reader allocate the same ref-slot index
for the `Collections$SetFromMap` and for every object in its subtree.
### What did you see instead?
`ClassCastException` on the read side, surfaced via
`ExceptionUtils.handleReadFailed` (a wrong-typed object returned from
`stream.readObject()` deep inside `readExternal`).
### Diagnosis
We instrumented the `ObjectOutput`/`ObjectInput` adapters used for
`Externalizable` serialization to log `[slotId, className]` for every
ref-tracked allocation, and ran a save+load cycle on the same data. Diffing the
two logs:
- The two streams agreed **perfectly** for the first ~74,343 allocations,
ending at the slot for the `Collections$SetFromMap` instance.
- Inside that Set's serialization (~1314 unlogged slots — non-Externalizable
Collection element loop dispatched through Fory's built-in
`SetFromMapSerializer`), the **reader allocated exactly one more slot than the
writer**.
- All subsequent slots were shifted by +1. Later back-references in the
stream then resolved one slot off, returning a wrong-typed object.
### Workaround
Forcing `Collections$SetFromMap` to use a user-registered
`CollectionSerializer` (one that writes/reads elements via plain `ctx.writeRef`
/ `ctx.readRef` and uses a `Collections.newSetFromMap(new
ConcurrentHashMap<>())` factory on the read side) makes the round-trip succeed.
So the issue is specific to the built-in `SetFromMapSerializer`'s `onMapWrite`
/ `onMapRead` ref-tracking accounting.
### Notes for triage
- #1832 / #1833 dealt with the same symptom shape; PR #1833 added an
`onMapWrite` fix. Worth checking whether that fix is reachable from the path
where the Set is encountered transitively via `stream.writeObject(...)` inside
a parent `writeExternal` (vs. the test cases in #1833 which serialize the Set
as a top-level argument).
- Happy to provide additional log output or test against a patched build if
useful — just don't have a small enough graph to share publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
