cshuo opened a new pull request, #19199: URL: https://github.com/apache/hudi/pull/19199
### Describe the issue this Pull Request addresses Table version 10 enables native log files by default, but table version 9 readers do not understand native log files. Downgrading a v10 MOR table to v9 must therefore ensure native data/delete logs are compacted away before the downgrade completes. Native CDC log conversion is intentionally out of scope for this downgrade path because v9 cannot read native `.cdc.parquet` files and converting them to legacy inline CDC logs would add broader CDC correctness and schema-evolution risk. ### Summary and Changelog - Added `TenToNineDowngradeHandler` to remove the v10-only `hoodie.table.storage.layout` table config during downgrade. - Wired `TEN -> NINE` downgrade routing in `UpgradeDowngrade` and added `(10, 9)` to the rollback-and-compact preflight set so native data/delete logs are compacted before v9 exposure. - Added native CDC fail-fast validation before rollback/compaction using native CDC filename detection. - Added `FSUtils.isNativeCDCLogFile(...)` support for identifying native CDC logs. - Added unit coverage for the 10-to-9 handler, route support, table config cleanup, and native CDC validation. - Added Spark downgrade coverage for v10 MOR tables with native logs, including validation that logs are compacted and storage layout config is removed. - Added Spark coverage for native CDC fail-fast behavior. - Removed temporary issue-19090 test workarounds that pinned writes to v9 now that 10-to-9 downgrade is supported. - Re-enabled Flink compactor downgrade coverage. - Extended existing current-plus-lower-version test matrices to include table version 9. ### Impact - **Functional impact**: Tables can now downgrade from version 10 to version 9 when native data/delete logs can be compacted away. Downgrade fails fast if native CDC logs are present. - **Maintainability**: Keeps v10-specific cleanup in a dedicated downgrade handler and reuses existing rollback-and-compaction downgrade preflight behavior. - **Extensibility**: Leaves native CDC downgrade conversion out of scope, preserving a clear future extension point if native-to-legacy CDC conversion is later designed. ### Risk Level medium. This touches core upgrade/downgrade routing and table-service preflight behavior, but the scope is limited to the new 10-to-9 downgrade path. Risk is mitigated by new handler tests, Spark integration coverage for native log compaction and CDC fail-fast, and cleanup of tests that previously avoided the unsupported downgrade path. ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
