Re: some questions, please help

[Yibo Cai]

Thanks Wes, Micah, your comments are very helpful.

Yibo

On 10/30/19 10:45 PM, Wes McKinney wrote:

On Wed, Oct 30, 2019 at 9:32 AM Micah Kornfield <emkornfi...@gmail.com> wrote:

- I see some SIMD optimizations in arrow go binding, such as vectored

sum. [2]

    But arrow cpp lib doesn't leverage SIMD. [3]
    Why not optimize it in cpp lib so all languages can benefit?

You're welcome to contribute such optimizations to the C++ library

Note that even though C++ doesn't use explicit SIMD intrinsics often times
the compiler will generate SIMD code because it can auto-vectorize the
code.

Note it will likely be important to have explicit dynamic/runtime SIMD
dispatching on certain hot paths as we build binaries that need to be
able to run on both newer and older CPUs

On Wed, Oct 30, 2019 at 7:25 AM Wes McKinney <wesmck...@gmail.com> wrote:

hi Yibo

On Wed, Oct 30, 2019 at 2:16 AM Yibo Cai <yibo....@arm.com> wrote:

Hi,

I'm new to Arrow. Would like to seek for help about some questions. Any

comment is welcomed.

- About source code tree, my understand is that "cpp" is the core arrow

libraries, "c_glib, go, python, ..." are language bindings to ease
integrating arrow into apps developed by that language. Is that correct?

No. We have 6 core implementations: C++, C#, Go, Java, JavaScript, and Rust

* C/GLib, MATLAB, Python, R bind to C++
* Ruby binds to GLib

- Arrow implements many data types and aggregation functions(sum, mean,

...). [1]

    IMO, more functions and types should be supported, like min/max,

vector/tensor operations, big number, etc. I'm not sure if this is in
arrow's scope, or the apps using arrow should deal with it themselves.

Our objective at least in the C++ library is to have a generally
useful "standard library" that handles common application concerns.
Whether or not something is thought to be in scope may vary on a case
by case basis -- if you can't find a JIRA issue for something in
particular, please go ahead and open one.

- I see some SIMD optimizations in arrow go binding, such as vectored

sum. [2]

    But arrow cpp lib doesn't leverage SIMD. [3]
    Why not optimize it in cpp lib so all languages can benefit?

You're welcome to contribute such optimizations to the C++ library


- Wes

[1]

https://github.com/apache/arrow/tree/master/cpp/src/arrow/compute/kernels

[2]

https://github.com/apache/arrow/blob/master/go/arrow/math/float64_avx2_amd64.s

[3]

https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/sum_internal.h#L99-L111

Yibo