yihua opened a new pull request, #19071: URL: https://github.com/apache/hudi/pull/19071
### Change Logs The native hudi-io HFile writer stored each block-index first key as a bare `[2-byte len][row]` with no HBase KeyValue suffix. An HBase-based HFile reader parsed it as a full KeyValue and read the family-length byte one past the end, throwing `ArrayIndexOutOfBoundsException` in `KeyValue.getFamilyLength` during a point or prefix lookup of a data-block boundary key. Full scans worked; only the block-index binary search crashed. Fix: `HFileRootIndexBlock` now writes each block-index first key as a full HBase KeyValue key (`[2-byte rowLen][row][cfLen=0][ts=LATEST][type=Put]`), matching the data-block entries. Backward compatibility: the native hudi-io reader is unchanged and reads both old bare-key and new full-key files (it compares only the row via the leading 2-byte length prefix), so no reader change and no format-version flag are needed. Only files written after this change are affected. Tests (hudi-io): - `TestHFileReadCompatibility`: an HBase reader point-looks-up every key (including block-boundary keys) in a native-written multi-block file; native-vs-HBase writer cells are byte-identical under the HBase reader; and a native file reads identically through the native and HBase readers. The point-lookup and byte-comparison cases run under both NONE and GZIP compression. - `TestHFileWriter`: a golden byte comparison of the data block and root block-index block locks the on-disk format; the single index entry grows by the 10-byte KeyValue suffix (4537 to 4547). ### Impact New metadata-table HFiles written by the native writer become point-lookup-readable by an HBase-based HFile reader. No change to the native read or write path or the on-disk layout beyond the 10-byte-per-index-entry KeyValue suffix. ### Risk level (write none, low medium or high below) low. The change is additive (a fixed suffix on block-index keys), covered by byte-exact and cross-reader tests under both compression modes, and the native reader reads both formats. ### Documentation Update None. ### Contributor's checklist - [ ] Read through contributor's guide - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
