RE: [C++] Runtime SIMD dispatching for Arrow

2020-09-08 Thread Du, Frank
Yes , best to has dedicated AVX512 device. Great news that you are working on the machine😊 Thanks, Frank -Original Message- From: Wes McKinney Sent: Monday, September 7, 2020 12:41 AM To: dev Subject: Re: [C++] Runtime SIMD dispatching for Arrow I might be able to contribute an AVX

Re: [C++] Runtime SIMD dispatching for Arrow

2020-09-06 Thread Wes McKinney
thub.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernel.h#L561 > [3] > https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/cpu_info.cc#L451 > > Thanks, > Frank > > -----Original Message- > From: Wes McKinney > Sent: Wednesday, May 13, 2020 9:39 PM

RE: [C++] Runtime SIMD dispatching for Arrow

2020-09-03 Thread Du, Frank
sage- From: Wes McKinney Sent: Wednesday, May 13, 2020 9:39 PM To: dev ; Micah Kornfield Subject: Re: [C++] Runtime SIMD dispatching for Arrow On Tue, May 12, 2020 at 11:12 PM Micah Kornfield wrote: > > > > > Since I develop on an AVX512-capable machine, if we have runtime

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-13 Thread Wes McKinney
On Tue, May 12, 2020 at 11:12 PM Micah Kornfield wrote: > > > > > Since I develop on an AVX512-capable machine, if we have runtime > > dispatching then it should be able to test all variants of a function > > from a single executable / test run rather than having to produce > > multiple builds and

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Micah Kornfield
> > Since I develop on an AVX512-capable machine, if we have runtime > dispatching then it should be able to test all variants of a function > from a single executable / test run rather than having to produce > multiple builds and test them separately, right? Yes, but I think the same of true with

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Wes McKinney
On Tue, May 12, 2020 at 9:47 PM Yibo Cai wrote: > > Thanks Wes, I'm glad to see this feature coming. > > From history talks, the main concern is runtime dispatcher may cause > performance issue. > Personally, I don't think it's a big problem. If we're using SIMD, it must be > targeting some tim

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Wes McKinney
On Tue, May 12, 2020 at 10:19 PM Micah Kornfield wrote: > > Hi Wes, > I think you highlighted the two issues well, but I think they are somewhat > orthogonal and runtime dispatching only addresses the binary availability > of the optimizations (but actually makes testing harder because it can > po

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Micah Kornfield
Hi Wes, I think you highlighted the two issues well, but I think they are somewhat orthogonal and runtime dispatching only addresses the binary availability of the optimizations (but actually makes testing harder because it can potentially hide untested code paths). Personally, I think it is valua

Re: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Yibo Cai
Thanks Wes, I'm glad to see this feature coming. From history talks, the main concern is runtime dispatcher may cause performance issue. Personally, I don't think it's a big problem. If we're using SIMD, it must be targeting some time consuming code. But we do need to take care some issues. E.

RE: [C++] Runtime SIMD dispatching for Arrow

2020-05-12 Thread Du, Frank
Hi, I totally agree that arrow should has a built-in support for runtime dispatching facilities just like other popular computing libs to fully utilize the modern hardware capacity, we feel arrow has great potential performance chance with the advanced cpu SIMD feature. It's ok for me to stop