DanielLeens commented on PR #11559: URL: https://github.com/apache/seatunnel/pull/11559#issuecomment-6594172845
Thanks @SEZ9, answering https://github.com/apache/seatunnel/pull/11559#issuecomment-6552600864 for PR11559-F1 (`KafkaSourceReader` deserialization). **Status of the file: not deferred, and unchanged since the fix commit.** The hardening landed in `5458405ede00c693da216f02c07908e4b11b11e7` (described in https://github.com/apache/seatunnel/pull/11559#issuecomment-6394093084). `git diff 5458405ede0 e7f159373393` is empty for both `KafkaSourceReader.java` and `KafkaSourceReaderGateTest.java`, and `5458405e` is an ancestor of `e7f15937`. The only `connector-kafka` changes that arrived in that range are dev's `KafkaPartitionSplitReader` / `KafkaPartitionSplitReaderTest` from the dev merge, which do not touch the gate path. The new head `bb23b9f27180b8c1b91d59a4a3977341a50ee321` (below) does not touch connector-kafka either, so you only need to re-check `KafkaSourceReader.java` at `5458405e` or any later head. **Deserializer shape on the current head** ([KafkaSourceReader.java#L398-L463](https://github.com/DanielLeens/seatunnel/blob/bb23b9f27180b8c1b91d59a4a3977341a50ee321/seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceReader.java#L398-L463)): - `deserializeSplit` constructs only `KafkaGateObjectInputStream`; it is the only `ObjectInputStream` construction in connector-kafka production sources. - `resolveClass` is an exact-name allow-list: `[B`, `String`, `Long`, `Integer`, `KafkaSourceSplit`, `TopicPartition`, `TablePath`. No prefix or package match; every other array descriptor is rejected. - `resolveProxyClass` throws before any interface or handler is resolved. - `restoreGateState` (L236-L246) checks each split's SHA-256 before calling `deserializeSplit`. - Negative tests in `KafkaSourceReaderGateTest`: `productionDeserializeSplitShouldRejectClassOutsideAllowlist`, `...RejectSamePackageSubclassOutsideAllowlist`, `...RejectArraysOutsideAllowlist`, `...RejectProxyClassesBeforeTheirHandler`, plus the positive `...RoundTripKafkaSourceSplit`. **Trust boundary.** The bytes come only from the engine's own checkpoint state. `snapshotGate` writes them, and `restoreGateState` reads them back during restore; there is no user- or network-facing input on this path. The SHA-256 digest is stored next to the payload, so it detects corruption but does not authenticate the payload: anyone who can write checkpoint storage can recompute it. The allow-list is therefore the real control. I relied on it rather than on the digest, and it is meant to hold even when the checkpoint store is not trusted. **CI on the previous head.** Fork run 41162787674 on `e7f15937` failed in `seatunnel-engine-server` testCompile: `SourceCheckpointLockHandoffTest.java:[565,20] cannot infer type arguments for SourceFlowLifeCycle<>`. The fixture merged from dev still used the old eight-argument constructor. `bb23b9f2` passes a default `SourceConfig` there and does not change the handoff behavior or assertions. This was Issue 5 of my November 1 review. Issues 6 and 7 from that review (dimension source ID after execution ID regeneration, and restore with two sources in one pipeline) are still open and not addressed by this push. CI for `bb23b9f2` has not finished yet, and I am not claiming it is green. <!-- daniel-11559-sez9-f1-20261102 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
