Re: [VOTE] Release Apache Arrow 0.7.0 - RC0

2017-09-14 Thread Wes McKinney
+1 (binding) I created a release verification script for Linux: https://github.com/apache/arrow/pull/1102 I also made a small fix to the Windows verification script: https://github.com/apache/arrow/pull/1101 On Linux (Ubuntu 14.04) I: * Verified GPG signature, checksums * Ran C++ unit tests (inc

[jira] [Created] (ARROW-1542) [C++] Windows release verification script should not modify conda environment

2017-09-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1542: --- Summary: [C++] Windows release verification script should not modify conda environment Key: ARROW-1542 URL: https://issues.apache.org/jira/browse/ARROW-1542 Project: Ap

[jira] [Created] (ARROW-1541) [C++] Race condition with arrow_gpu

2017-09-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1541: --- Summary: [C++] Race condition with arrow_gpu Key: ARROW-1541 URL: https://issues.apache.org/jira/browse/ARROW-1541 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-1540) [C++] Fix valgrind warnings in cuda-test if possible

2017-09-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1540: --- Summary: [C++] Fix valgrind warnings in cuda-test if possible Key: ARROW-1540 URL: https://issues.apache.org/jira/browse/ARROW-1540 Project: Apache Arrow Issue

[jira] [Created] (ARROW-1539) [C++] Remove functions deprecated as of 0.7.0 and prior releases

2017-09-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1539: --- Summary: [C++] Remove functions deprecated as of 0.7.0 and prior releases Key: ARROW-1539 URL: https://issues.apache.org/jira/browse/ARROW-1539 Project: Apache Arrow

Re: Java Examples Writing Flatbuffer in IPC Message

2017-09-14 Thread Wes McKinney
hi Andrew, Are you talking about writing a stream to shared memory? There are not functions in the Arrow Java library to do this out of the box. This would be a great contribution to make to the project: https://issues.apache.org/jira/browse/ARROW-721 If you create an OutputStream that writes to

Re: Java Examples Writing Flatbuffer in IPC Message

2017-09-14 Thread Wes McKinney
1. Are you able to write to an shared memory IPC segment as an OutputStream in Java? If so, do that, and use either the ArrowFileWriter or ArrowStreamWriter in Java to write to it, like https://github.com/apache/arrow/blob/master/java/tools/src/main/java/org/apache/arrow/tools/Integration.java#L160

Re: Java Examples Writing Flatbuffer in IPC Message

2017-09-14 Thread Andrew Pham (BLOOMBERG/ 731 LEX)
Thanks for the reply Wes. I'm just looking for a way to ferry structured data (like a row in a SQL table) from one process (Java) to another (C++). Any ideas on how to execute this? From: dev@arrow.apache.org At: 09/14/17 18:32:12To: Andrew Pham (BLOOMBERG/ 731 LEX ) , dev@arrow.apache.org

Java Examples Writing Flatbuffer in IPC Message

2017-09-14 Thread Andrew Pham (BLOOMBERG/ 731 LEX)
The receiver/reader would be a C++ process. I'm looking for simple examples that illustrate this IPC. Can anyone point me to any? Thanks!

Re: spark error with reading parquet file created vis pandas/pyarrow

2017-09-14 Thread Wes McKinney
The option pq.write_table(..., flavor='spark') made it into the 0.7.0 release - Wes On Fri, Sep 8, 2017 at 6:28 PM, Julien Le Dem wrote: > The int96 deprecation is slowly bubbling up the stack. There are still > discussions in spark on how to make the change. So for now even though it's > de

[jira] [Created] (ARROW-1538) [C++] Support Ubuntu 14.04 in .deb packaging automation

2017-09-14 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1538: --- Summary: [C++] Support Ubuntu 14.04 in .deb packaging automation Key: ARROW-1538 URL: https://issues.apache.org/jira/browse/ARROW-1538 Project: Apache Arrow Is