peterxcli opened a new pull request, #5135: URL: https://github.com/apache/datafusion-comet/pull/5135
## Which issue does this PR close? Closes #5059. ## Rationale for this change `CalendarIntervalType` is accepted by the Scala hash-expression compatibility check, but the shared native hasher did not support Arrow's `Interval(MonthDayNano)` representation. As a result, native `hash` and `xxhash64` expressions failed at runtime instead of matching Spark. ## What changes are included in this PR? - Add shared native hashing for `Interval(MonthDayNano)`, covering both Murmur3 and xxHash64. - Convert Arrow nanoseconds back to Spark microseconds and match Spark's generated hash code, including its omission of the days field. - Add a focused regression test covering both hash functions, nulls, negative intervals, and days-only intervals. ## How are these changes tested? - `make core` - `./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometHashExpressionSuite calendar interval" -Dscalastyle.skip=true` (1 test passed) - `cargo fmt --manifest-path native/Cargo.toml --all --check` - `cargo clippy --manifest-path native/Cargo.toml -p datafusion-comet-spark-expr --lib --tests -- -D warnings` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
