neilconway commented on PR #20553:
URL: https://github.com/apache/datafusion/pull/20553#issuecomment-3961617195
So it turns out that when using `ryu`, the output differs in a few cosmetic
ways from how Rust's default formatting displays floats. For example, ryu
returns "1.0" for the floating point number `1`, whereas previously we returned
"1". There will presumably be other differences in various other FP corner
cases (e.g., scientific notation).
For the time being, I've reverted the change to use `ryu`; we now display
floating point numbers using `write!`. That is significantly slower (roughly
2x) for floats, but still much better than the original implementation. I'm
curious if anyone has a point of view on whether using `ryu` is the right
approach here or not.
Updated benchmark numbers:
```
┌──────────────┬────────────┬─────────┬───────────┬─────────────┐
│ Benchmark │ Array Size │ Main │ Optimized │ Improvement │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ int64 │ 5 │ 229 µs │ 72 µs │ -69% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ int64 │ 20 │ 546 µs │ 159 µs │ -71% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ int64 │ 100 │ 1.94 ms │ 620 µs │ -68% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ float64 │ 5 │ 617 µs │ 266 µs │ -57% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ float64 │ 20 │ 1.91 ms │ 909 µs │ -52% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ float64 │ 100 │ 8.17 ms │ 4.32 ms │ -47% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ string │ 5 │ 254 µs │ 65 µs │ -75% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ string │ 20 │ 532 µs │ 128 µs │ -76% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ string │ 100 │ 1.79 ms │ 486 µs │ -73% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ nested int64 │ 5 │ 206 µs │ 81 µs │ -60% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ nested int64 │ 20 │ 468 µs │ 166 µs │ -65% │
├──────────────┼────────────┼─────────┼───────────┼─────────────┤
│ nested int64 │ 100 │ 1.83 ms │ 626 µs │ -66% │
└──────────────┴────────────┴─────────┴───────────┴─────────────┘
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
