goutamadwant opened a new pull request, #12776: URL: https://github.com/apache/seatunnel/pull/12776
### Purpose of this pull request Fix #9063. CDC sources exchange source events between readers and the enumerator. On Spark they ran through the parallel source path, which rejects these events with `UnsupportedOperationException ... SupportCoordinate`. - `IncrementalSource` now implements `SupportCoordinate`, so Spark runs one enumerator for all readers in a single task. Zeta only references the marker in a TODO comment and Flink does not use it. - Coordinated source path fixes: report the configured parallelism (was 0), fail the task on reader or construction errors instead of hanging, and run an enumerator-only checkpoint every second so the snapshot phase completes at parallelism > 1. No reader offsets are committed. - Micro-batch (`readStream`) reads of coordinated sources are rejected with a clear error, since state is not restored between micro-batches. The marker-only change in #4269 is not enough on its own: it hangs at parallelism > 1 and hides reader errors. ### Does this PR introduce _any_ user-facing change? Yes. MySQL-CDC jobs run on Spark 2.4 and 3.3 instead of failing at the first finished snapshot split. Spark limits are documented in en and zh: no checkpoint restore, sinks that flush only on checkpoints need `batch_size`, and Postgres-CDC does not advance the replication slot. Adding the interface changes the computed `serialVersionUID` of `IncrementalSource`. ### How was this patch tested? | Test | Result | |---|---| | `CoordinatedSourceTest`, `CoordinatedBatchPartitionReaderTest`, `IncrementalSourceEnumeratorTest` (real enumerator at parallelism 2), JDK 11 and 17 | Fail before, pass after | | `MysqlCDCIT` on Zeta, Flink 1.18, Flink 1.20, Spark 3.3 | 36/37; the Flink 1.20 timeout passed 3/3 on rerun | | New Spark cases on Spark 2.4.6 and 3.3.0, parallelism 1 and 2: seed rows, check the snapshot, then a binlog insert, update and delete | 4/4 | | `Mysql8_4CDCIT` shared methods | 12/12 | Five shared base tests stay disabled on Spark because their JDBC sink flushes only on checkpoints. ### Check list - [x] No new runtime dependencies (`seatunnel-translation-base` added to `connector-cdc-base` in test scope). - [x] Updated English and Chinese documentation. - [x] Zeta and Flink behaviour unchanged; no incompatible-changes entry. - [x] Added E2E coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
