andygrove opened a new pull request, #359: URL: https://github.com/apache/datafusion-spark/pull/359
## Which issue does this PR close? - Closes #358. ## Rationale for this change `expm1` called Rust's `f64::exp_m1`, which uses the platform's C math library. Spark's `Expm1` calls Java's `StrictMath.expm1`, which returns fdlibm's result on every platform. On macOS the results differ in the last bit for some inputs, and `math/expm1.slt` fails: | Query | Spark 4.2.0 and this PR | Before, on macOS | |---|---|---| | `expm1(1)` | `1.718281828459045` | `1.7182818284590453` | | `expm1(-2.5D)` | `-0.9179150013761013` | `-0.9179150013761012` | | `expm1(0.9D)` | `1.4596031111569499` | `1.4596031111569496` | | `expm1(1.7D)` | `4.473947391727199` | `4.4739473917272` | ## What changes are included in this PR? `expm1` calls `libm::expm1`. The `libm` crate is a pure Rust port of musl's math library, which comes from fdlibm, so it gives Spark's result on every platform. `libm` was already in `Cargo.lock` as a transitive dependency, and is now a direct dependency of `datafusion-spark`. ## Are these changes tested? Yes. New records in `math/expm1.slt` cover the three inputs above, small and large inputs, and overflow to `Infinity`. The existing `expm1(1)` record and the new ones for `-2.5`, `0.9` and `1.7` fail on macOS without this change. I checked `expm1.slt` against Spark 4.2.0 with the PySpark runner, and every record matches. I also compared `libm::expm1` with Spark for 19 more inputs, including `-0.0`, `NaN` and both infinities, and every result matches. ## Are there any user-facing changes? `expm1` returns Spark's result on every platform. Linux results do not change for the inputs tested. This PR was prepared with AI assistance (Claude Code) and reviewed by me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
