hi Yibo, I agree with this, having #ifdef in many places in the codebase is not maintainable longer-term.
As far as runtime dispatch, we could populate a function table of all machine-dependent functions once so then the dispatch isn't happening on each function. Or some similar strategy This of course presumes that functions with runtime SIMD dispatch do not need to be inlined. For functions that need to be inlined, a different approach may be required. - Wes On Thu, Mar 19, 2020 at 9:57 PM Yibo Cai <yibo....@arm.com> wrote: > > I'm revisiting this old thread as I see some avx512 code merged recently[1]. > Code maintenance will be non-trivial if we want to cover more > hardware(sse/avx/avx512/neon/sve/...) and optimize more code in the future. > #ifdef is obviously no-go. > > So I'm selling my proposal again :) > - put all machine dependent code in one place (similar to what linux manages > various cpu arches) > - add runtime dispatcher to select best simd code snippet per running hardware > > I can provide a PR for community review first. Thoughts? > > [1] https://github.com/apache/arrow/pull/6650 > > On 2019/12/24 18:17:25, Wes McKinney <w...@gmail.com> wrote: > > If we go the route of AOT-compilation of Gandiva kernels as an> > > approach to generate a shared library with many kernels, we might> > > indeed look at possibly generating a "fat" binary with runtime> > > dispatch between AVX2-optimized vs. SSE <= 4.2 (or non-SIMD> > > altogether) kernels. This is something we could do during the code> > > generation step where we generate the "stubs" to invoke the IR> > > kernels.> > > > > Given where the project is at in its development trajectory, it seems> > > important to come up with some concrete answers to some of these> > > questions to reduce developer anxiety that may otherwise prevent> > > forward progress in feature development.> > > > > On Tue, Dec 24, 2019 at 2:37 AM Micah Kornfield <em...@gmail.com> wrote:> > > >> > > > I would lean against adding another library dependency. My main concerns> > > > with adding another library dependency are:> > > > 1. Supporting it across all of the build tool-chains (using a GCC > > > specific> > > > option would be my least favorite approach).> > > > 2. Distributed binary size (for wheels at least people seem to care).> > > >> > > > I would like lean more towards yes if there were some real world > > > benchmarks> > > > showing the a substantial performance gain.> > > >> > > > I don't think it is unreasonable to package our binaries targeting a > > > common> > > > instruction set (e.g. AVX 1 or 2). For those that want to make full use > > > of> > > > their latest hardware compiling from source doesn't seem unreasonable,> > > > especially given the recent effort to trim dependencies.> > > >> > > > Cheers,> > > > Micah> > > >> > > >> > > >> > > > On Fri, Dec 20, 2019 at 2:13 AM Antoine Pitrou <an...@python.org> wrote:> > > >> > > > >> > > > > Hi,> > > > >> > > > > I would recommend against reinventing the wheel. It would be possible> > > > > to reuse an existing C++ SIMD library. There are several of them (Vc,> > > > > xsimd, libsimdpp...). Of course, "just use Gandiva" is another > > > > possible> > > > > answer.> > > > >> > > > > Regards> > > > >> > > > > Antoine.> > > > >> > > > >> > > > > Le 20/12/2019 à 08:32, Yibo Cai a écrit :> > > > > > Hi,> > > > > >> > > > > > I'm investigating SIMD support to C++ compute kernel(not gandiva).> > > > > >> > > > > > A typical case is the sum kernel[1]. Below tight loop can be easily> > > > > optimized with SIMD.> > > > > >> > > > > > for (int64_t i = 0; i < length; i++) {> > > > > > local.sum += values[i];> > > > > > }> > > > > >> > > > > > Compiler already does loop vectorization. But it's done at compile > > > > > time> > > > > without knowledge of target cpu.> > > > > > Binaries compiled with avx-512 cannot run on old cpu, while binaries> > > > > compiled with only sse4 enabled is suboptimal on new hardware.> > > > > >> > > > > > I have some proposals, would like to hear comments from community.> > > > > >> > > > > > - Based on our experience of ISA-L[2] project(optimized storage> > > > > acceleration library for x86 and Arm), runtime dispatcher is a good> > > > > approach. Basically, it links in codes optimized for different cpu> > > > > features(sse4,avx2,neon,...) and selects the best one fits target cpu > > > > at> > > > > first invocation. This is similar to gcc indirect function[3], but > > > > doesn't> > > > > depend on compilers.> > > > > >> > > > > > - Use gcc FMV [4] to generate multiple binaries for one function. See> > > > > sample source and compiled code [5].> > > > > > Though looks simple, it has many limitations: It's gcc specific> > > > > feature, no support from clang and msvc. It only works on x86, no Arm> > > > > support.> > > > > > I think this approach is no-go.> > > > > >> > > > > > - Don't do it.> > > > > > Gandiva leverages LLVM JIT for runtime code optimization. Is it> > > > > duplicated effort to do it in C++ kernel? Will these vetorizable> > > > > computations move to Gandiva in the future?> > > > > >> > > > > > [1]> > > > > https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/sum_internal.h#L104-L106> > > > > > [2] https://github.com/intel/isa-l> > > > > > [3] https://willnewton.name/2013/07/02/using-gnu-indirect-functions/> > > > > > [4] https://lwn.net/Articles/691932/> > > > > > [5] https://godbolt.org/z/ajpuq_> > > > > >> > > > >> > >
