201811510411lw opened a new pull request, #12366: URL: https://github.com/apache/seatunnel/pull/12366
### Purpose of this pull request Address the fixed-bucket routing bug in #12243. Normal Paimon jobs are wrapped in `MultiTableSink`, so inspecting only the per-table sink capability does not activate routing on the actual execution path. Parallel writers can then write conflicting changes to the same physical bucket. Schema-control rows also need a destination independent of data conversion. This PR combines the original partitioner/SPI work from [#12301](https://github.com/apache/seatunnel/pull/12301) by zhangshenghang with the wrapper, schema-control, and wiring-test fixes discussed in [DanielLeens's review](https://github.com/apache/seatunnel/pull/12301#issuecomment-5678515912). The original contribution is credited with a co-author trailer. The branch contains one commit, `5de24b8cb`, based on Apache `dev` at `1650a04cd`: - Enable the existing `SupportSinkDataPartition` / `SinkDataPartitioner` capability through the actual `MultiTableSink` and resolve each table's policy once. Reject mixed routing, unsupported replicas, and duplicate physical targets. - Use `PaimonFixedBucketPartitioner` with Paimon's `FixedBucketWriteSelector` to route fixed buckets to their owner. - Route zero-field schema-control rows by `schema_subtask_id` without passing them through row conversion. Expand broadcasts before routing and changing parallelism. - Exercise the real factory/processor path with a Flink MiniCluster and Paimon readback, including different source/sink parallelism, multiple tables/buckets, checkpoint-boundary changes, and control-row delivery. Extend the existing Paimon E2E suite. This is limited to routing and its regression coverage. It excludes global-commit recovery, new checkpoint-state serializers, and writer/committer rework. ### Does this PR introduce _any_ user-facing change? Yes. Flink automatically activates routing for `HASH_FIXED` Paimon tables, including wrapped single-table jobs. Routed multi-table sinks require `multi_table_sink_replica = 1`; mixed routed/unrouted sinks and independent source writers targeting the same physical table fail at initialization. EN/ZH documentation and incompatibility notes describe these constraints. The partitioner still uses its initial schemas. Safe schema-control delivery does not imply full online structural DDL support. This PR adds no commit-recovery or rescaling guarantees. ### How was this patch tested? Local verification of this branch on JDK 8 / Maven 3.8.8: - Clean verify of the affected reactor: 83 tests, zero failures/errors/skips, including all 10 MiniCluster routing cases. - CI specification checks: 34 tests, zero failures/errors, 2 skips. All four Markdown checks passed. - Spotless apply/check passed; packaged routing classes matched the compiled classes. - Updated `PaimonIT` compiled. Docker E2E was not run locally. Commands: ```bash mvn -B \ -pl seatunnel-connectors-v2/connector-paimon,seatunnel-core/seatunnel-flink-starter/seatunnel-flink-13-starter,seatunnel-core/seatunnel-flink-starter/seatunnel-flink-15-starter,seatunnel-core/seatunnel-flink-starter/seatunnel-flink-20-starter \ -am -Dskip.spotless=true -DskipIT=true -Dmaven.jar.forceCreation=true \ -DskipUT=false -DskipTests=false -Dmaven.test.skip=false \ -Dsurefire.failIfNoSpecifiedTests=false \ -Dtest=FlinkSinkWriterTest,MultiTableSinkAggregatedCommitterTest,MultiTableSinkCommitterTest,MultiTableSinkTest,MultiTableSinkWriterSchemaChangeBroadcastTest,MultiTableSinkWriterTest,MultiTableWriteRoutingTest,PaimonBucketAssignerTest,PaimonFixedBucketPartitionerTest,PaimonFlinkBucketRoutingTest,PaimonWriteTest,SchemaOperatorTest,SinkWriteRoutingPartitionerTest,SupportSinkDataPartitionTest \ clean verify mvn -B -pl seatunnel-ci-tools -am clean test \ -Dlicense.skipAddThirdParty=true --no-snapshot-updates ``` Full remote CI is pending; these local results are not a claim that all CI jobs have passed. ### Check list - [x] EN/ZH connector and developer documentation updated. - [x] Incompatibility notes updated. - [x] Existing Paimon E2E suite extended; no new connector registration is needed. - [x] No binary JAR files added; MiniCluster dependencies are test-scoped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
