Re: [C++] Revamping approach to Arrow compute kernel development

2020-04-18 Thread Wes McKinney
I started a brain dump of some issues that come to mind around kernel implementation and array expression evaluation. I'll try to fill this out, and it would be helpful to add supporting citations to other projects about what kinds of issues come up and what implementation strategies may be helpful

[jira] [Created] (ARROW-8514) [Developer] Windows wheel verification script does not check Python 3.5

2020-04-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8514: --- Summary: [Developer] Windows wheel verification script does not check Python 3.5 Key: ARROW-8514 URL: https://issues.apache.org/jira/browse/ARROW-8514 Project: Apache A

[jira] [Created] (ARROW-8513) [Python] Expose Take with Table input in Python

2020-04-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8513: --- Summary: [Python] Expose Take with Table input in Python Key: ARROW-8513 URL: https://issues.apache.org/jira/browse/ARROW-8513 Project: Apache Arrow Issue Type

[NIGHTLY] Arrow Build Report for Job nightly-2020-04-18-2

2020-04-18 Thread Crossbow
Arrow Build Report for Job nightly-2020-04-18-2 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-2 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-2-github-centos-8-amd64 - debian-buster-a

[C++] Revamping approach to Arrow compute kernel development

2020-04-18 Thread Wes McKinney
hi folks, This e-mail comes in the context of two C++ data processing subprojects we have discussed in the past * Data Frame API https://docs.google.com/document/d/1XHe_j87n2VHGzEbnLe786GHbbcbrzbjgG8D0IXWAeHg/edit * In-memory Query Engine https://docs.google.com/document/d/10RoUZmiMQRi_J1FcPeVAUA

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-18 Thread Wes McKinney
Right, sorry about that. So the difference it turns out is the UNITY build. The compiler error in ARROW-8510 does not occur for some reason when CMAKE_UNITY_BUILD=ON. On Sat, Apr 18, 2020 at 3:36 PM Antoine Pitrou wrote: > > > We build with both VS2015 and VS2017 on AppVeyor. > > Regards > > Ant

[jira] [Created] (ARROW-8512) [C++] Delete unused compute expr prototype code

2020-04-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8512: --- Summary: [C++] Delete unused compute expr prototype code Key: ARROW-8512 URL: https://issues.apache.org/jira/browse/ARROW-8512 Project: Apache Arrow Issue Type

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-18 Thread Antoine Pitrou
We build with both VS2015 and VS2017 on AppVeyor. Regards Antoine. Le 18/04/2020 à 22:34, Wes McKinney a écrit : > The release verification script (which uses the VS 2017 compiler mode) > fails for me on Windows > > https://issues.apache.org/jira/browse/ARROW-8510 > > It looks like we are b

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-18 Thread Wes McKinney
The release verification script (which uses the VS 2017 compiler mode) fails for me on Windows https://issues.apache.org/jira/browse/ARROW-8510 It looks like we are building with VS 2019 in CI. We may want to be more conservative and build with VS 2015 or VS 2017 in CI. This is peculiar though be

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-18 Thread Neal Richardson
+1 (binding) I ran verification on Ubuntu, macOS, and Windows using crossbow. Thanks to all who helped debug. A few things are still failing: see https://github.com/apache/arrow/pull/6961#issuecomment-615940240 for details. Since Python folks are unconcerned by the wheel issues I encountered, I wo

[jira] [Created] (ARROW-8511) [Developer][Release] Windows release verification script does not halt if C++ compilation fails

2020-04-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8511: --- Summary: [Developer][Release] Windows release verification script does not halt if C++ compilation fails Key: ARROW-8511 URL: https://issues.apache.org/jira/browse/ARROW-8511

[jira] [Created] (ARROW-8510) [Release][C++] Dataset unit tests are reported as failed in arrow-verify-release-candidate.bat

2020-04-18 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8510: --- Summary: [Release][C++] Dataset unit tests are reported as failed in arrow-verify-release-candidate.bat Key: ARROW-8510 URL: https://issues.apache.org/jira/browse/ARROW-8510

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-04-18-1

2020-04-18 Thread Wes McKinney
With continued "No space left on device" issues on GHA, what does that indicate as far as long-term prospects for GitHub Actions? The amount of binary artifacts that are builds produce and the amount of third party code they depend on is going to continue to increase. On Sat, Apr 18, 2020 at 12:13

[NIGHTLY] Arrow Build Report for Job nightly-2020-04-18-1

2020-04-18 Thread Crossbow
Arrow Build Report for Job nightly-2020-04-18-1 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-1 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-1-github-centos-8-amd64 - debian-buster-a

0.17 release blog post: help needed

2020-04-18 Thread Neal Richardson
Hi all, Since it looks like we're close to releasing 0.17, we need to fill in the details for our blog post announcement. I've started a document here: https://docs.google.com/document/d/16UKZtvL49o8nCDN8JU3Ut6y76Y9d8-4qXv5vFv7aNvs/edit#heading=h.kqqacbm2lpv8 Please fill in the details for the par

[jira] [Created] (ARROW-8509) GArrowRecordBatch <-> GArrowBuffer conversion functions

2020-04-18 Thread Tanveer (Jira)
Tanveer created ARROW-8509: -- Summary: GArrowRecordBatch <-> GArrowBuffer conversion functions Key: ARROW-8509 URL: https://issues.apache.org/jira/browse/ARROW-8509 Project: Apache Arrow Issue Type:

Re: [jira] [Created] (ARROW-8508) [Rust] ListBuilder of FixedSizeListBuilder creates wrong offsets

2020-04-18 Thread Lee Nelson
unsubscribe On Sat, Apr 18, 2020 at 5:43 AM Christian Beilschmidt (Jira) < j...@apache.org> wrote: > Christian Beilschmidt created ARROW-8508: > > > Summary: [Rust] ListBuilder of FixedSizeListBuilder creates > wrong offsets >

[jira] [Created] (ARROW-8508) [Rust] ListBuilder of FixedSizeListBuilder creates wrong offsets

2020-04-18 Thread Christian Beilschmidt (Jira)
Christian Beilschmidt created ARROW-8508: Summary: [Rust] ListBuilder of FixedSizeListBuilder creates wrong offsets Key: ARROW-8508 URL: https://issues.apache.org/jira/browse/ARROW-8508 Projec

Re: Thoughts and issues regarding the release procedure

2020-04-18 Thread Sutou Kouhei
Hi, Krisztián, thanks for collecting the problems! > *PREPARE_CHANGELOG* phase: > > - need to set ARROW_HOME because changelog.py requires it https://github.com/apache/arrow/pull/6975 > - changelog.py stopped working since adding support for parquet tickets [1], > because it requires the actua

[jira] [Created] (ARROW-8507) [Release] Detect .git directory automatically in changelog.py

2020-04-18 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-8507: --- Summary: [Release] Detect .git directory automatically in changelog.py Key: ARROW-8507 URL: https://issues.apache.org/jira/browse/ARROW-8507 Project: Apache Arrow

[jira] [Created] (ARROW-8506) [c++] Miss tests to verify expected_buffer with bit_width > 8 in RLE

2020-04-18 Thread Kazuaki Ishizaki (Jira)
Kazuaki Ishizaki created ARROW-8506: --- Summary: [c++] Miss tests to verify expected_buffer with bit_width > 8 in RLE Key: ARROW-8506 URL: https://issues.apache.org/jira/browse/ARROW-8506 Project: Apa

Re: [VOTE] Release Apache Arrow 0.17.0 - RC0

2020-04-18 Thread Sutou Kouhei
+1 (binding) I ran the followings on Debian GNU/Linux sid: * TEST_CSHARP=0 \ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 \ CUDA_TOOLKIT_ROOT=/usr \ ARROW_CMAKE_OPTIONS=-DgRPC_SOURCE=BUNDLED \ dev/release/verify-release-candidate.sh source 0.17.0 0 * dev/release/verif

[NIGHTLY] Arrow Build Report for Job nightly-2020-04-18-0

2020-04-18 Thread Crossbow
Arrow Build Report for Job nightly-2020-04-18-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-0 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-18-0-github-centos-8-amd64 - debian-buster-a