neilconway commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4106763514
On an M4 Max, it looks like the crossover point between direct iteration and
using the Arrow kernel is 32-40 list elements:
```
┌───────────┬──────────┬──────────┬─────────────────────┐
│ List size │ Scalar │ Kernel │ Kernel vs Scalar │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 8 │ 54.8 µs │ 172.7 µs │ scalar 3.2x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 16 │ 105.3 µs │ 188.1 µs │ scalar 1.8x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 32 │ 232.5 µs │ 253.2 µs │ scalar 1.09x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 48 │ 362.6 µs │ 329.6 µs │ kernel 1.10x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 64 │ 492.8 µs │ 444.2 µs │ kernel 1.11x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 96 │ 761.7 µs │ 589.0 µs │ kernel 1.29x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 128 │ 1.032 ms │ 782.0 µs │ kernel 1.32x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 256 │ 2.076 ms │ 1.428 ms │ kernel 1.45x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 512 │ 4.138 ms │ 2.728 ms │ kernel 1.52x faster │
└───────────┴──────────┴──────────┴─────────────────────┘
```
So I lowered the iteration -> kernel switchover threshold to 32.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
