nsivabalan opened a new pull request, #18378:
URL: https://github.com/apache/hudi/pull/18378
### Describe the issue this Pull Request addresses
Fixes an issue with partitioned Record Level Index (RLI) when using nested
partition fields containing dots (e.g., fare.currency) combined with Hive-style
partitioning. The partition path encoding was not handling dots properly,
causing issues with partition path to file ID mapping in the metadata table.
### Summary and Changelog
Summary: Enables partitioned RLI to work correctly with nested partition
fields that contain dots when using Hive-style partitioning.
Changes:
- Added '.' (dot) character to the escape list in
PartitionPathEncodeUtils.java:59 so partition paths like "fare.currency=USD" are
properly encoded when stored in the metadata table
- Added comprehensive functional test
testPartitionedRecordLevelIndexWithHiveStylePartitioningAndDotInPartitionField()
in
TestRecordLevelIndex.scala:679-746 that validates:
- Initial inserts with dot-containing partition field work correctly
- Record index entries are properly created and retrievable
- Updates/upserts continue to work after the initial write
- All 10 records are correctly indexed and queryable
### Impact
This fix enables users to use partitioned RLI with nested column names
(containing dots) as partition fields when Hive-style
partitioning is enabled. Without this fix, such configurations would
result in incorrect partition path encoding in the metadata table.
No breaking changes to existing functionality - this is purely a bug fix
for a specific edge case.
### Risk Level
low
### Documentation Update
<!-- Describe any necessary documentation update if there is any new
feature, config, or user-facing change. If not, put "none".
- The config description must be updated if new configs are added or the
default value of the configs are changed.
- Any new feature or user-facing change requires updating the Hudi website.
Please follow the
[instruction](https://hudi.apache.org/contribute/developer-setup#website)
to make changes to the website. -->
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
