Hi folks, I'm looking for reviewers for PR #17774, which speeds up ZOrderByteUtils.interleaveBits by roughly 10x: https://github.com/apache/iceberg/pull/17774
The current implementation moves one bit per loop iteration, and each iteration searches for the next source column, with four Z-ordered columns at 8 bytes each, that is 256 hard-to-pipeline iterations per row, and it sits on the hot path of every `rewrite_data_files` with the Z-order strategy. When every column contributes the same number of bytes (which is what all callers produce), interleaving is a fixed permutation that depends only on the source byte and the column count, so it can be tabulated. SPREAD(n,b) holds the bits of byte b spread n positions apart; OR-ing the spread bytes of n columns, each shifted by its column index, yields the n output bytes at once. On JMH (10M rows per op, JDK 17, i7-13700H) the gain is consistent across shapes (lower is better): two columns go from 3.67 to 0.46 s/op, three columns from 5.60 to 0.69, four columns from 7.38 to 0.79, and the four-column case with 8-byte output from 1.83 to 0.26. Since interleaving is the per-row combined step of the Z-order rewrite path, this should translate into a meaningful reduction of rewrite_data_files Z-order runtime. For reviewers who want to verify the bit permutation, there is a fully worked 3-column numerical example with every intermediate value in the PR: https://github.com/apache/iceberg/pull/17774#issuecomment-5382755566 The PR already has one approval; I'd appreciate a committer taking a look at the approach and the table construction. Thanks! Gianluca
