nzw921rx opened a new issue, #12779: URL: https://github.com/apache/seatunnel/issues/12779
### Search before asking - [X] I searched in the [issues](https://github.com/apache/seatunnel/issues) and found nothing similar. ### SeaTunnel Version dev (any version that contains #11095) ### Project Version _No response_ ### Minimal Reproduction `FilterFieldTransformFactory.optionRule()` declares `include_fields` / `exclude_fields` as an **exclusive required** group, so one of them must be configured at the transform level: ```java return OptionRule.builder() .exclusive( FilterFieldTransformConfig.INCLUDE_FIELDS, FilterFieldTransformConfig.EXCLUDE_FIELDS) .optional( FilterFieldTransformConfig.INCLUDE_FIELDS, Conditions.notEmpty(FilterFieldTransformConfig.INCLUDE_FIELDS)) .optional( FilterFieldTransformConfig.EXCLUDE_FIELDS, Conditions.notEmpty(FilterFieldTransformConfig.EXCLUDE_FIELDS)) .optional(TransformCommonOptions.MULTI_TABLES) .optional(TransformCommonOptions.TABLE_MATCH_REGEX) .optional(TransformCommonOptions.RULE_MATCH_MODE) .build(); ``` `OptionRule.Builder#exclusive(...)` maps to `RequiredOption.ExclusiveRequiredOptions`, and `ConfigValidator#checkExclusive` rejects the configuration when none of the group is set: ``` options: include_fields, exclude_fields type: exclusive constraint: exactly one option must be set, but none are configured ``` This is incompatible with the multi-table mode, where the per-table rules intentionally live inside `table_transform` entries and the transform level sets neither `include_fields` nor `exclude_fields`: ```hocon transform { Filter { plugin_input = "fake" plugin_output = "fake1" table_transform = [{ table_path = "test.xyz" include_fields = [name] }] } } ``` ### Steps to reproduce 1. Use the `Filter` transform with `table_transform` as above (per-table rules only). 2. Submit the job. ### Expected Behavior The job starts. `test.xyz` keeps only `name`; other tables are unchanged (same for a `table_transform` entry that uses `exclude_fields`). ### Actual Behavior The job fails deterministically before any data is processed: ``` org.apache.seatunnel.api.configuration.util.OptionValidationException: options: include_fields, exclude_fields type: exclusive constraint: exactly one option must be set, but none are configured ``` `FactoryUtil#createAndPrepareMultiTableTransform` runs `ConfigValidator.of(context.getOptions()).validate(factory.optionRule())` on the transform-level options, so a valid multi-table configuration never reaches `FilterFieldMultiCatalogTransform`. ### Additional context - Regression from #11095 ("Migrate transforms validation to declarative OptionRule"); before that commit neither option was required, so multi-table rules worked. - `FilterFieldTransformFactoryTest#testNeitherFieldsFails` encodes the same assumption (config without either option must throw), which is why the multi-table case has no coverage. - The same `exclusive(...)` pattern was introduced for `CopyFieldTransformFactory` (`.exclusive(CopyTransformConfig.FIELDS, CopyTransformConfig.SRC_FIELD)`) by the same commit, and `Copy` also supports `table_transform` (see `docs/en/transforms/transform-multi-table.md`), so it most likely fails the same way. ### Suggested fix - Keep `include_fields` / `exclude_fields` optional, and reject "both configured" through a `ConditionExtension` (`Conditions.extension(...)`). - Enforce "at least one of them must be configured" only for the non-multi-table path, for example inside the transform (like `TextChunkTransformFactory` does with its own `validate(...)`), or by using `.conditional(...)` / `.conditionalRule(...)` on `table_transform`. - Add a regression test that validates the option rule with only `table_transform` configured, and one that builds the transform end-to-end for a multi-table configuration. ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
