zephyr45 commented on PR #13060: URL: https://github.com/apache/seatunnel/pull/13060#issuecomment-6523497874
Thanks @DanielLeens — both review points are addressed in `10e618cb`. - Recovery now inspects every assigned `KafkaSinkState`, derives and deduplicates each saved transaction namespace, and fences all unfinished transactions before opening new work. For current-format state it validates the engine's modulo assignment (`restoredWriterIndex % currentParallelism == currentWriterIndex`) before any Kafka producer is initialized, so a live writer cannot fence a sibling. Legacy shared-prefix checkpoints remain supported. - The writer unit coverage now includes a 3-to-2 scale-down (old writers 0 and 2 assigned to new writer 0) and fail-fast rejection of an incorrectly assigned namespace. The streaming Zeta E2E now restores from parallelism 3 to 2, sends data through all three source partitions, retains a single-partition sink, and passed against Kafka 7.0.9. - Added focused Javadocs for the checkpointed job prefix and writer-owned transaction namespace. Post-revision validation: all 139 Kafka connector unit tests passed; both affected real-broker E2Es passed; the 16-module affected dependency reactor packaged successfully; Spotless, `git diff --check`, and the English/Chinese documentation checks passed. For @SEZ9's failure-status question, the PR description now records that current `dev` already propagates asynchronous Kafka send/fencing failures at the next write or `prepareCommit`, causing task/checkpoint failure rather than a silently successful lossy commit; engine retry policy determines final job status. That propagation predates this PR. The silent `FINISHED` behavior was reproduced on released 2.3.13. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
