carloea2 opened a new pull request, #8186: URL: https://github.com/apache/texera/pull/8186
### What changes were proposed in this PR? Add Java-compatible hashing for Python `LARGE_BINARY` tuple fields. The URI is hashed as UTF-16 code units, then wrapped with the same `Objects.hash` seed used by Scala `LargeBinary`. Before: a hash shuffle on a large-binary key raised `KeyError` before choosing a worker. After: `s3://bucket/object` hashes to -46745592 and selects worker 0 with two receivers. ### Any related issues, documentation, discussions? Closes #8185 ### How was this PR tested? The new regression was red before the source change for both ASCII and Unicode URIs, then both passed after the mapping was added. Null values continue to use the existing zero hash. ```text python -c "import sys,pytest; sys.path[:0]=[r'<worktree>\amber\src\main\python',r'<checkout>\amber\src\main\python']; raise SystemExit(pytest.main([r'amber/src/test/python/core/models/test_tuple.py::TestTuple::test_hash_supports_large_binary','-q','-p','no:cacheprovider']))" ``` The model layer and partitioner suites pass: ```text python -c "import sys,pytest; sys.path[:0]=[r'<worktree>\amber\src\main\python',r'<checkout>\amber\src\main\python']; raise SystemExit(pytest.main([r'amber/src/test/python/core/models','-k','not test_hash','-q','-p','no:cacheprovider']))" python -c "import sys,pytest; sys.path[:0]=[r'<worktree>\amber\src\main\python',r'<checkout>\amber\src\main\python']; raise SystemExit(pytest.main([r'amber/src/test/python/core/architecture/sendsemantics/test_partitioners.py','-q','-p','no:cacheprovider']))" ``` Results: 313 model tests passed with one expected failure, and all 35 partitioner tests passed. Hash-named fixtures were run separately because the existing `TestTuple.test_hash` uses a naive epoch timestamp that fails on Windows; PR #8174 makes it explicitly UTC. ```text ruff check amber/src/main/python amber/src/test/python ruff format --check amber/src/main/python amber/src/test/python ``` Ruff checked 213 files successfully. JDK 17 independently verified both expected hashes. The fixed production hash partitioner selected receiver A instead of raising. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex, GPT-5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
