Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Kouhei Sutou
The GPG key I used was my old weak GPG key (DSA 1024bit key). So it's not suitable to add the KEYS file... In "Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)" on Thu, 4 Oct 2018 06:28:30 -0400, Wes McKinney wrote: > Can you add your other GPG key to the KEYS file? If it's a trusted key > tha

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Wes McKinney
In principle these aren't dealbreakers. Since the project has been growing a lot lately, I'm not surprised to see some rough edges. * After the release we should probably downgrade gbenchmark and report the issue upstream * The Plasma failure is definitely odd, I haven't seen any such thing in Tra

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Uwe L. Korn
I'm struggling a bit to verify the release with the following issues: * CMake is too old on Ubuntu 16.04, gbenchmark needs a newer one. A workaround here would be to install a newer one with conda. Should we do that always? * Java tests fail on Ubuntu 16.04: https://issues.apache.org/jira/browse

[jira] [Created] (ARROW-3443) [Java] Flight reports memory leaks in TestBasicOperation

2018-10-04 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-3443: -- Summary: [Java] Flight reports memory leaks in TestBasicOperation Key: ARROW-3443 URL: https://issues.apache.org/jira/browse/ARROW-3443 Project: Apache Arrow Iss

[jira] [Created] (ARROW-3442) [C++] Use dynamic linking for unit tests, ensure coverage working properly with clang

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3442: --- Summary: [C++] Use dynamic linking for unit tests, ensure coverage working properly with clang Key: ARROW-3442 URL: https://issues.apache.org/jira/browse/ARROW-3442 Pro

[jira] [Created] (ARROW-3441) [Gandiva][C++] Produce fewer test executables

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3441: --- Summary: [Gandiva][C++] Produce fewer test executables Key: ARROW-3441 URL: https://issues.apache.org/jira/browse/ARROW-3441 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3440) [Gandiva][C++] Remove outdated cpp/src/gandiva/README.md, add build documentation to cpp/README.md

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3440: --- Summary: [Gandiva][C++] Remove outdated cpp/src/gandiva/README.md, add build documentation to cpp/README.md Key: ARROW-3440 URL: https://issues.apache.org/jira/browse/ARROW-3440

[jira] [Created] (ARROW-3439) [R] R language bindings for Feather format

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3439: --- Summary: [R] R language bindings for Feather format Key: ARROW-3439 URL: https://issues.apache.org/jira/browse/ARROW-3439 Project: Apache Arrow Issue Type: New

[jira] [Created] (ARROW-3438) [Packaging] Crossbow changelog generator escapes markdown bulletpoint

2018-10-04 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-3438: -- Summary: [Packaging] Crossbow changelog generator escapes markdown bulletpoint Key: ARROW-3438 URL: https://issues.apache.org/jira/browse/ARROW-3438 Project: Apac

Re: Gandiva snapshot releases

2018-10-04 Thread Wes McKinney
Appveyor is running against master; there was an issue with the status hooks that was fixed recently On Thu, Oct 4, 2018 at 8:02 AM Wes McKinney wrote: > > hi Praveen, > > Probably the best way to accomplish this is to use our new Crossbow > infrastructure for task automation on Travis CI and Appv

Re: Gandiva snapshot releases

2018-10-04 Thread Wes McKinney
hi Praveen, Probably the best way to accomplish this is to use our new Crossbow infrastructure for task automation on Travis CI and Appveyor rather than trying to do all of this within the CI entries. This is how we are producing all of our binary artifacts for releases now -- presumably in future

Gandiva snapshot releases

2018-10-04 Thread Praveen Kumar
Hi Folks, As part of https://issues.apache.org/jira/browse/ARROW-3385, we are planning to perform a snapshot release of the Gandiva Jar on each commit to master. This would be a platform independent jar that contains the core gandiva library and its jni bridge packaged for Mac, Windows and *nix p

Re: [JAVA] Arrow performance measurement

2018-10-04 Thread Wes McKinney
See e.g. https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/ipc-read-write-test.cc#L222 On Thu, Oct 4, 2018 at 6:48 AM Animesh Trivedi wrote: > > Primarily write the same microbenchmark as I have in Java in C++ for table > reading and value materialization. So just an example of equi

Re: [JAVA] Arrow performance measurement

2018-10-04 Thread Animesh Trivedi
Primarily write the same microbenchmark as I have in Java in C++ for table reading and value materialization. So just an example of equivalent ArrowFileReader example code in C++. Unit tests are a good starting point, thanks for the tip :) On Thu, Oct 4, 2018 at 12:39 PM Wes McKinney wrote: > >

Re: [JAVA] Arrow performance measurement

2018-10-04 Thread Wes McKinney
> 3. Are there examples of Arrow in C++ read/write code that I can have a look? What kind of code are you looking for? I would direct you to relevant unit tests that exhibit certain functionality, but it depends on what you are trying to do On Wed, Oct 3, 2018 at 9:45 AM Animesh Trivedi wrote: >

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Wes McKinney
Can you add your other GPG key to the KEYS file? If it's a trusted key that seems OK to me On Thu, Oct 4, 2018 at 4:34 AM Kouhei Sutou wrote: > > Hi, > > Oh... > > I've resign by correct GPG key and replaced .asc files. > > Can we use this RC? Or should we create one more RC for > fixing this .asc

[jira] [Created] (ARROW-3437) [Gandiva][C++] Configure static linking of libgcc, libstdc++ with LDFLAGS

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3437: --- Summary: [Gandiva][C++] Configure static linking of libgcc, libstdc++ with LDFLAGS Key: ARROW-3437 URL: https://issues.apache.org/jira/browse/ARROW-3437 Project: Apach

Re: Using CUDA enabled pyarrow

2018-10-04 Thread Wes McKinney
hi Pearu -- yes, I had thought of this work working on the arrow_gpu library. Some time ago I opened https://issues.apache.org/jira/browse/ARROW-1470 thinking that it would be good to combine the MemoryPool* concept and the AllocateBuffer concept into a single abstract interface. Such an interfac

[jira] [Created] (ARROW-3433) [C++] Validate re2 with Windows toolchain, EP

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3433: --- Summary: [C++] Validate re2 with Windows toolchain, EP Key: ARROW-3433 URL: https://issues.apache.org/jira/browse/ARROW-3433 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3436) [C++] Boost version required by Gandiva is too new for Ubuntu 14.04

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3436: --- Summary: [C++] Boost version required by Gandiva is too new for Ubuntu 14.04 Key: ARROW-3436 URL: https://issues.apache.org/jira/browse/ARROW-3436 Project: Apache Arrow

[jira] [Created] (ARROW-3435) [C++] Add option to use dynamic linking with re2

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3435: --- Summary: [C++] Add option to use dynamic linking with re2 Key: ARROW-3435 URL: https://issues.apache.org/jira/browse/ARROW-3435 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-3434) [Packaging] Add Apache ORC C++ library to conda-forge

2018-10-04 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3434: --- Summary: [Packaging] Add Apache ORC C++ library to conda-forge Key: ARROW-3434 URL: https://issues.apache.org/jira/browse/ARROW-3434 Project: Apache Arrow Issu

Re: Using CUDA enabled pyarrow

2018-10-04 Thread Pearu Peterson
Hi, Currently, the arrow host memory management includes MemoryPool to accelerate memory operations (new/free). Would there be interest in supporting the same concept in CUDA memory management to reduce the overhead of cudaMalloc/cudaFree? Best regards, Pearu On Wed, Oct 3, 2018 at 11:44 PM Pearu

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Kouhei Sutou
Hi, Oh... I've resign by correct GPG key and replaced .asc files. Can we use this RC? Or should we create one more RC for fixing this .asc problem? Thanks, -- kou In <1538639225.4190225.1530323248.542da...@webmail.messagingengine.com> "Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)" on Thu, 0

[jira] [Created] (ARROW-3432) [Packaging] Variables aren't expanded Subversion commit message

2018-10-04 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3432: --- Summary: [Packaging] Variables aren't expanded Subversion commit message Key: ARROW-3432 URL: https://issues.apache.org/jira/browse/ARROW-3432 Project: Apache Arrow

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Uwe L. Korn
Hello Kou, It seems like you have used a GPG key that is not in the main keys files: ``` + gpg --verify apache-arrow-0.11.0.tar.gz.asc apache-arrow-0.11.0.tar.gz gpg: Signature made Thu Oct 4 05:46:23 2018 CEST gpg:using DSA key 7714A383F6F73E2D9828791D17423F641C837F31 gpg: Can't

Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)

2018-10-04 Thread Kouhei Sutou
+1 (binding) I ran the followings on Debian GNU/Linux sid: * ARROW_HAVE_GPU=no dev/release/verify-release-candidate.sh source 0.11.0 1 * dev/release/verify-release-candidate.sh binaries 0.11.0 1 with: * gcc version 8.2.0 (Debian 8.2.0-1) * OpenJDK 1.8.0_181 * Ruby 2.6.0dev * NodeJS