[DISCUSS][Java] Builders for java classes

2019-10-23 Thread Micah Kornfield
As part a PR Ji Liu has made to help populate data for test cases [1], the question came up on whether we should provide a more builder classes in java for ValueVectors. The proposed implementation would wrap the existing Writer classes. Do people think this would be a valuable addition to the j

[jira] [Created] (ARROW-6983) [C++] Threaded task group crashes sometimes

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6983: -- Summary: [C++] Threaded task group crashes sometimes Key: ARROW-6983 URL: https://issues.apache.org/jira/browse/ARROW-6983 Project: Apache Arrow Issue Ty

[VOTE] Clarifications and forward compatibility changes for Dictionary Encoding

2019-10-23 Thread Micah Kornfield
Hello, As discussed on [1], I've proposed clarifications in a PR [2] that clarifies: 1. It is not required that all dictionary batches occur at the beginning of the IPC stream format (if a the first record batch has an all null dictionary encoded column, the null column's dictionary might not be

Re: [DISCUSS] Result vs Status

2019-10-23 Thread Micah Kornfield
OK, it sounds like people want Result (at least in some circumstances). Any thoughts on migrating old APIs and what to do for new APIs going forward? A very rough approximation [1] yields the following counts by module: 853 arrow 17 gandiva 25 parquet 50 plasma [1] grep -r Status cpp

Re: [DISCUSS][Java] Design of the algorithm module

2019-10-23 Thread Micah Kornfield
> > To save the effort, or invest it to higher priority issues, we plan to: > 1. We will stop providing "additional algorithms", unless they are > explictly required. This sounds reasonable, we can also evaluate on a case-by-case basis on how widely applicable some are. 2. For existing addition a

Re: [C++] The quest for zero-dependency builds

2019-10-23 Thread Micah Kornfield
I'll add I don't think we will actually be switching anytime soon. bazel does have some advantages at least over our current CMake system in terms of developer productivity (users can target smaller components with unit tests which avoid re linking). I've started on a prototype and hope to have s

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-23-0

2019-10-23 Thread Krisztián Szűcs
It was happening from time to time, but now it is pretty consistent. I'm working on to fix the deployments by running the crossbow artifact uploading script. On Thu, Oct 24, 2019 at 1:16 AM Wes McKinney wrote: > Any clues why the macOS wheel uploads keep flaking out? > > On Wed, Oct 23, 2019 at

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-23-0

2019-10-23 Thread Wes McKinney
Any clues why the macOS wheel uploads keep flaking out? On Wed, Oct 23, 2019 at 7:56 AM Crossbow wrote: > > > Arrow Build Report for Job nightly-2019-10-23-0 > > All tasks: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-23-0 > > Failed Tasks: > - docker-clang-format:

Re: [C++] The quest for zero-dependency builds

2019-10-23 Thread Wes McKinney
On Sun, Oct 20, 2019 at 12:22 PM Maarten Ballintijn wrote: > > Dev's > > I would request to be as conservative as possible in choosing (keeping) a > build system. > > For developers, packagers and even end-users for some languages the build > system is just > another dependency. Even if cmake is

[jira] [Created] (ARROW-6982) [R] Add bindings for compare and boolean kernels

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6982: -- Summary: [R] Add bindings for compare and boolean kernels Key: ARROW-6982 URL: https://issues.apache.org/jira/browse/ARROW-6982 Project: Apache Arrow Iss

[jira] [Created] (ARROW-6981) [R] Implement HDFS file-system interface in R

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6981: -- Summary: [R] Implement HDFS file-system interface in R Key: ARROW-6981 URL: https://issues.apache.org/jira/browse/ARROW-6981 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6980) [R] dplyr backend for RecordBatch/Table

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6980: -- Summary: [R] dplyr backend for RecordBatch/Table Key: ARROW-6980 URL: https://issues.apache.org/jira/browse/ARROW-6980 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6979) [R] Enable jemalloc in autobrew formula

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6979: -- Summary: [R] Enable jemalloc in autobrew formula Key: ARROW-6979 URL: https://issues.apache.org/jira/browse/ARROW-6979 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6978) [R] Add bindings for sum and mean compute kernels

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6978: -- Summary: [R] Add bindings for sum and mean compute kernels Key: ARROW-6978 URL: https://issues.apache.org/jira/browse/ARROW-6978 Project: Apache Arrow Is

[jira] [Created] (ARROW-6977) [C++] Only enable jemalloc background_thread if feature is supported

2019-10-23 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6977: -- Summary: [C++] Only enable jemalloc background_thread if feature is supported Key: ARROW-6977 URL: https://issues.apache.org/jira/browse/ARROW-6977 Project: Apach

[jira] [Created] (ARROW-6976) Possible memory leak in pyarrow read_parquet

2019-10-23 Thread david cottrell (Jira)
david cottrell created ARROW-6976: - Summary: Possible memory leak in pyarrow read_parquet Key: ARROW-6976 URL: https://issues.apache.org/jira/browse/ARROW-6976 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-6975) [C++] Put make_unique in its own header

2019-10-23 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6975: - Summary: [C++] Put make_unique in its own header Key: ARROW-6975 URL: https://issues.apache.org/jira/browse/ARROW-6975 Project: Apache Arrow Issue Type: Wi

[jira] [Created] (ARROW-6974) [C++] Implement Cast kernel for time-likes with ArrayDataVisitor pattern

2019-10-23 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-6974: Summary: [C++] Implement Cast kernel for time-likes with ArrayDataVisitor pattern Key: ARROW-6974 URL: https://issues.apache.org/jira/browse/ARROW-6974

[NIGHTLY] Arrow Build Report for Job nightly-2019-10-23-0

2019-10-23 Thread Crossbow
Arrow Build Report for Job nightly-2019-10-23-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-23-0 Failed Tasks: - docker-clang-format: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-23-0-circle-docker-clang-format - docke

[jira] [Created] (ARROW-6973) [C++][ThreadPool] Use perfect forwarding in Submit

2019-10-23 Thread Artem Alekseev (Jira)
Artem Alekseev created ARROW-6973: - Summary: [C++][ThreadPool] Use perfect forwarding in Submit Key: ARROW-6973 URL: https://issues.apache.org/jira/browse/ARROW-6973 Project: Apache Arrow Iss