Le 01/04/2022 à 08:43, Sasha Krassovsky a écrit :
I agree that a potential inconsistent experience is a problem, but I disagree that SIMD would be the root of the problem, or even be a significant contributor to it.
It would be a very significant contributor, as the inconsistency can manifest under the form of up to 8-fold differences in performance (or perhaps more). Usually, compiler differences can produce differences on the order of a few tens of percent, and that is understood by users, but an order of magnitude is unexpected and makes it more difficult to advertise Arrow performance.
And lastly, who can say that xsimd will be compiled properly and perform better than autovectorization?
I don't think such theoretical statements help a lot. There is concrete proof that autovectorization produces very flimsy results (even on the same compiler, simply by varying the datatypes). Is there any proof that xsimd-using code produces such fragile results? I haven't seen any, and it is unlikely to be the case (why would a compiler deoptimize SIMD intrinsics into plain scalar code?).
At the very least, for an initial PR, I would like to keep everything simpler.
We can certainly do that. There is a far cry however, between the proposal of leveraging autovectorization as a first step towards better performance, and the original proposal of removing the xsimd dependency and only relying on autovectorization for future efforts :-)
Regards Antoine.
