Hi Wes,
On 10/30/19 10:24 PM, Wes McKinney wrote:
hi Yibo
On Wed, Oct 30, 2019 at 2:16 AM Yibo Cai <yibo....@arm.com> wrote:
Hi,
I'm new to Arrow. Would like to seek for help about some questions. Any comment
is welcomed.
- About source code tree, my understand is that "cpp" is the core arrow libraries,
"c_glib, go, python, ..." are language bindings to ease integrating arrow into apps
developed by that language. Is that correct?
No. We have 6 core implementations: C++, C#, Go, Java, JavaScript, and Rust
* C/GLib, MATLAB, Python, R bind to C++
* Ruby binds to GLib
I wonder how arrow deals with gaps among different implementations? Say, C++
lib implements some features go lib doesn't support. Is there a consistent API
document, or documents for each language implementation?
- Arrow implements many data types and aggregation functions(sum, mean, ...).
[1]
IMO, more functions and types should be supported, like min/max,
vector/tensor operations, big number, etc. I'm not sure if this is in arrow's
scope, or the apps using arrow should deal with it themselves.
Our objective at least in the C++ library is to have a generally
useful "standard library" that handles common application concerns.
Whether or not something is thought to be in scope may vary on a case
by case basis -- if you can't find a JIRA issue for something in
particular, please go ahead and open one.
- I see some SIMD optimizations in arrow go binding, such as vectored sum. [2]
But arrow cpp lib doesn't leverage SIMD. [3]
Why not optimize it in cpp lib so all languages can benefit?
You're welcome to contribute such optimizations to the C++ library
- Wes
[1] https://github.com/apache/arrow/tree/master/cpp/src/arrow/compute/kernels
[2]
https://github.com/apache/arrow/blob/master/go/arrow/math/float64_avx2_amd64.s
[3]
https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/sum_internal.h#L99-L111
Yibo
