carloea2 opened a new issue, #8251: URL: https://github.com/apache/texera/issues/8251
### Current behavior Java `Double` hashing canonicalizes all NaN bit patterns, while Python tuple hashing preserves the raw NaN payload. Noncanonical NaNs therefore produce different tuple hashes in the Python and JVM engines and can route the same hash-shuffle key to different workers. Before: signaling NaN payload hashes to 2146435104 in Python and 2146959391 with Java semantics After: every NaN hashes to the Java canonical value 2146959391 ### Steps to reproduce On `50321e403c82df299a13deb50a7f9849dd93bdba`, create a double from bits `7ff0000000000001` and hash a tuple containing it. Observed values: ```text payload_bits=7ff0000000000001 python_tuple_hash=2146435104 java_compatible_tuple_hash=2146959391 ``` ### Expected behavior Python should canonicalize NaN bits before applying the Java double hash so shuffle routing matches the JVM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
