Re: [jira] [Created] (ARROW-6294) [C++] Use hyphen for plasma-store-server executable

2019-08-19 Thread Zhuo Jia Dai
how do I unsubscribe? On Tue, Aug 20, 2019 at 3:18 PM Sutou Kouhei (Jira) wrote: > Sutou Kouhei created ARROW-6294: > --- > > Summary: [C++] Use hyphen for plasma-store-server executable > Key: ARROW-6294 > URL: https

[jira] [Created] (ARROW-6294) [C++] Use hyphen for plasma-store-server executable

2019-08-19 Thread Sutou Kouhei (Jira)
Sutou Kouhei created ARROW-6294: --- Summary: [C++] Use hyphen for plasma-store-server executable Key: ARROW-6294 URL: https://issues.apache.org/jira/browse/ARROW-6294 Project: Apache Arrow Issue

Re: [VOTE] Proposed addition to Arrow Flight Protocol

2019-08-19 Thread Micah Kornfield
The motion carries with 4 binding +1 votes, 2 non-binding +1 votes and no other votes. I think the next step is to review and merge the patch pending patch [1]. Thanks, Micah [1] https://github.com/apache/arrow/pull/4980 On Mon, Aug 19, 2019 at 2:52 AM Antoine Pitrou wrote: > > +1 (bindin

[jira] [Created] (ARROW-6293) datafusion 0.15.0-SNAPSHOT error

2019-08-19 Thread xingzhicn (Jira)
xingzhicn created ARROW-6293: Summary: datafusion 0.15.0-SNAPSHOT error Key: ARROW-6293 URL: https://issues.apache.org/jira/browse/ARROW-6293 Project: Apache Arrow Issue Type: Bug Compo

[jira] [Created] (ARROW-6292) [C++] Add an option to build with mimalloc

2019-08-19 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6292: - Summary: [C++] Add an option to build with mimalloc Key: ARROW-6292 URL: https://issues.apache.org/jira/browse/ARROW-6292 Project: Apache Arrow Issue Type:

Re: [Discuss] [Python] protocol for conversion to pyarrow Array

2019-08-19 Thread Wes McKinney
No concerns from me either. On Mon, Aug 19, 2019 at 5:10 AM Antoine Pitrou wrote: > > > No concern from me. It should probably be documented somewhere though :-) > > Regards > > Antoine. > > > Le 16/08/2019 à 17:23, Joris Van den Bossche a écrit : > > Coming back to this older thread, I have ope

Having a hard time merging PRs after the JIRA upgrade

2019-08-19 Thread Wes McKinney
It seems that INFRA upgraded JIRA -- over the last hour the JIRA REST API seems to be struggling, I'm not sure what's going wrong: https://issues.apache.org/jira/browse/INFRA-18900 - Wes

Re: [C++] Naming changes merged

2019-08-19 Thread Wes McKinney
Thanks for doing this! Perhaps we should document these conventions in developer/cpp.rst in the Sphinx project On Mon, Aug 19, 2019 at 12:13 PM Antoine Pitrou wrote: > > > Hello, > > In https://github.com/apache/arrow/pull/5069 I've merged the naming > changes previously discussed. In short: > -

Re: Timeline for 0.15.0 release

2019-08-19 Thread Ji Liu
Hi, Wes, on the java side, I can think of several bugs that need to be fixed or reminded. i. ARROW-6040: Dictionary entries are required in IPC streams even when empty[1] This one is under review now, however through this PR we find that there seems a bug in java reading and writing dictionaries

[C++] Naming changes merged

2019-08-19 Thread Antoine Pitrou
Hello, In https://github.com/apache/arrow/pull/5069 I've merged the naming changes previously discussed. In short: - always use underscores in C++ source file names - always use underscores in .so / .a file names - always use hyphens in executable file names - always use hyphens in pkgconfig fi

Re: Timeline for 0.15.0 release

2019-08-19 Thread Wes McKinney
I'm going to work some on organizing the 0.15.0 backlog some this week, if anyone wants to help with grooming (particularly for languages other than C++/Python where I'm focusing) that would be helpful. There have been almost 500 JIRA issues opened since the 0.14.0 release, so we should make sure t

Re: [DISCUSS] ArrayBuilders with mutable type

2019-08-19 Thread Antoine Pitrou
If it becomes much more expensive then calling it `type()` (rather than e.g. GetCurrentType()) is a bit misleading. Regards Antoine. Le 19/08/2019 à 16:27, Wes McKinney a écrit : > On Mon, Aug 19, 2019 at 9:16 AM Ben Kietzman wrote: >> >> Thanks for responding. >> >> I can certainly add Visi

Re: [DISCUSS] ArrayBuilders with mutable type

2019-08-19 Thread Wes McKinney
On Mon, Aug 19, 2019 at 9:16 AM Ben Kietzman wrote: > > Thanks for responding. > > I can certainly add VisitBuilder/VisitBuilderInline for ArrayBuilder, but > there's a slight difficulty: some types don't have a single concrete > builder class. For example, the builders of dictionary arrays are te

Re: [DISCUSS] ArrayBuilders with mutable type

2019-08-19 Thread Francois Saint-Jacques
Indeed, I'd expect the `type()` method to not be called in the hot path. François On Mon, Aug 19, 2019 at 10:17 AM Wes McKinney wrote: > > hi Ben, > > On this possibility > > - Make ArrayBuilder::type() virtual. This will be much more expensive for > nested builders and for applications which ne

Re: [DISCUSS] ArrayBuilders with mutable type

2019-08-19 Thread Wes McKinney
hi Ben, On this possibility - Make ArrayBuilder::type() virtual. This will be much more expensive for nested builders and for applications which need to branch on ArrayBuilder::type()->id() ArrayBuilder::type_id() should be provided as well. Could you explain what would be a situation where thes

Re: [DISCUSS] ArrayBuilders with mutable type

2019-08-19 Thread Ben Kietzman
Thanks for responding. I can certainly add VisitBuilder/VisitBuilderInline for ArrayBuilder, but there's a slight difficulty: some types don't have a single concrete builder class. For example, the builders of dictionary arrays are templated on the encoded type and also on an index builder, which

Re: [DISCUSS] Apache Arrow manylinux1 support

2019-08-19 Thread Wes McKinney
On Mon, Aug 19, 2019 at 8:55 AM Antoine Pitrou wrote: > > On Mon, 19 Aug 2019 08:44:26 -0500 > Wes McKinney wrote: > > Will publishing only manylinux2010 wheels have any consequences (for > > example, a relatively new version of setuptools may be required)? > > A relatively new version of pip is

Re: [DISCUSS] Apache Arrow manylinux1 support

2019-08-19 Thread Antoine Pitrou
On Mon, 19 Aug 2019 08:44:26 -0500 Wes McKinney wrote: > Will publishing only manylinux2010 wheels have any consequences (for > example, a relatively new version of setuptools may be required)? A relatively new version of pip is required. But upgrading pip is straightforward, at least in a virtu

Re: [DISCUSS] Apache Arrow manylinux1 support

2019-08-19 Thread Wes McKinney
Will publishing only manylinux2010 wheels have any consequences (for example, a relatively new version of setuptools may be required)? On Fri, Aug 16, 2019 at 11:58 AM Neal Richardson wrote: > > For R's official support for various C++ versions, see > https://cran.r-project.org/doc/manuals/r-rele

Re: Gandiva Java benchmarks

2019-08-19 Thread Ravindra Pindikura
On Sat, Aug 17, 2019 at 5:09 AM Rui Wang wrote: > I got help for a pointer to Gandiva cpp's micro benchmark > < > https://github.com/apache/arrow/blob/master/cpp/src/gandiva/tests/micro_benchmarks.cc > >. > I will start from there. > There is also a primitive java variant here : https://github.

Re: [Discuss] [Python] protocol for conversion to pyarrow Array

2019-08-19 Thread Antoine Pitrou
No concern from me. It should probably be documented somewhere though :-) Regards Antoine. Le 16/08/2019 à 17:23, Joris Van den Bossche a écrit : > Coming back to this older thread, I have opened a PR with a proof of > concept of the proposed protocol to convert third-party array objects to

Re: [DISCUSS][Format][C++] Improvement of sparse tensor format and implementation

2019-08-19 Thread Antoine Pitrou
Hi, This sounds fine on the principle. I'll let other comment on the details. Regards Antoine. Le 19/08/2019 à 11:29, Kenta Murata a écrit : > Hi, > > I’d like to propose the following improvement of the sparse tensor > format and implementation. > > (1) To make variable bit-width indices

Re: [VOTE] Proposed addition to Arrow Flight Protocol

2019-08-19 Thread Antoine Pitrou
+1 (binding) Regards Antoine. Le 16/08/2019 à 07:44, Micah Kornfield a écrit : > Hello, > Ryan Murray has proposed adding a GetFlightSchema RPC [1] to the Arrow > Flight Protocol [2]. The purpose of this RPC is to allow decoupling schema > and endpoint retrieval as provided by the GetFlightI

[DISCUSS][Format][C++] Improvement of sparse tensor format and implementation

2019-08-19 Thread Kenta Murata
Hi, I’d like to propose the following improvement of the sparse tensor format and implementation. (1) To make variable bit-width indices available. The main purpose of the first part of the proposal is making 32-bit indices available. It allows us to serialize scipy.sparse.csr_matrix objects et