DanielLeens commented on issue #12361: URL: https://github.com/apache/seatunnel/issues/12361#issuecomment-5714101191
Thanks for the detailed evidence. **Classification: A / Connector-V2 CDC snapshot correctness and memory safety.** The 2.3.13 source confirms the three material facts in this report: the splitter plans its terminal range as `ChunkRange.of(chunkStart, null)`; an exactly-once snapshot forces the Debezium queue size to `Integer.MAX_VALUE`; and `pollSplitRecordsIfExactlyOnce()` retains records in a `LinkedHashMap` until the end watermark before materializing the output list. Therefore `snapshot.split.size` guides enumeration but is not a memory bound for the terminal split. There is currently no supported configuration that makes this exact-once terminal path memory-bounded. Increasing heap only moves the failure point. Please do not treat `exactly_once = false` as a general mitigation, because that changes the delivery contract. Until a code change exists, the only conservative operational mitigation is to keep write growth during the initial snapshot within the available memory budget, for example by scheduling a controlled write-maintenance window; that is not a configuration fix. A fix needs a deterministic CDC-base regression, not just a larger heap: plan a terminal split, grow the table after enumeration, force a checkpoint/restore, and prove bounded retained records plus no lost or duplicate snapshot/binlog rows. It must establish the correct watermark boundary rather than assuming that a planning-time maximum key is safe. Please also attach the sanitized `SHOW CREATE TABLE`, full source split options, and the GC/heap evidence around the failure so the test data shape matches the incident. No implementation claim is recorded yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
