[jira] [Created] (ARROW-2122) Pyarrow fails to serialize dataframe with timestamp.

2018-02-08 Thread Robert Nishihara (JIRA)
Robert Nishihara created ARROW-2122: --- Summary: Pyarrow fails to serialize dataframe with timestamp. Key: ARROW-2122 URL: https://issues.apache.org/jira/browse/ARROW-2122 Project: Apache Arrow

[jira] [Created] (ARROW-2121) Consider special casing object arrays in pandas serializers.

2018-02-08 Thread Robert Nishihara (JIRA)
Robert Nishihara created ARROW-2121: --- Summary: Consider special casing object arrays in pandas serializers. Key: ARROW-2121 URL: https://issues.apache.org/jira/browse/ARROW-2121 Project: Apache Arro

[jira] [Created] (ARROW-2120) Add possibility to use empty _MSVC_STATIC_LIB_SUFFIX for Thirdparties

2018-02-08 Thread rip.nsk (JIRA)
rip.nsk created ARROW-2120: -- Summary: Add possibility to use empty _MSVC_STATIC_LIB_SUFFIX for Thirdparties Key: ARROW-2120 URL: https://issues.apache.org/jira/browse/ARROW-2120 Project: Apache Arrow

Re: [Python] Retrieving a RecordBatch from plasma inside a function

2018-02-08 Thread Philipp Moritz
Thanks! I can indeed reproduce this problem. I'm a bit busy right now and plan to look into it on the weekend. Here is the preliminary backtrace for everybody interested: CESS (code=1, address=0x38158) frame #0: 0x00010e6457fc lib.so`__pyx_pw_7pyarrow_3lib_10Int32Value_1as_py(_object

[jira] [Created] (ARROW-2119) Handle Arrow stream with zero record batch

2018-02-08 Thread Jingyuan Wang (JIRA)
Jingyuan Wang created ARROW-2119: Summary: Handle Arrow stream with zero record batch Key: ARROW-2119 URL: https://issues.apache.org/jira/browse/ARROW-2119 Project: Apache Arrow Issue Type: B

Re: Memory mapping error on pq.read_table

2018-02-08 Thread Wes McKinney
hi Simba, is it possible the file has zero length? $ touch foo $ ipython In [1]: import pyarrow In [2]: pyarrow.memory_map('foo') --- ArrowIOError Traceback (most recent call last) in () >

[jira] [Created] (ARROW-2118) [Python] Improve error message when calling parquet.read_table on an empty file

2018-02-08 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2118: --- Summary: [Python] Improve error message when calling parquet.read_table on an empty file Key: ARROW-2118 URL: https://issues.apache.org/jira/browse/ARROW-2118 Project:

[jira] [Created] (ARROW-2117) [C++] Pin clang to version 5.0

2018-02-08 Thread Phillip Cloud (JIRA)
Phillip Cloud created ARROW-2117: Summary: [C++] Pin clang to version 5.0 Key: ARROW-2117 URL: https://issues.apache.org/jira/browse/ARROW-2117 Project: Apache Arrow Issue Type: Improvement

Memory mapping error on pq.read_table

2018-02-08 Thread simba nyatsanga
Hi Everyone, I've encountered a memory mapping error when attempting to read a parquet file to a Pandas DataFrame. It seems to be happening intermittently though, I've so far encountered it once. In my case the pq.read_table code is being invoked in a Linux docker container. I had a look at the do

Re: Delta dictionaries: implementation

2018-02-08 Thread Brian Hulette
Glad to see someone is interested in dictionary deltas! The Javascript implementation does handle deltas, but we only have an arrow reader implementation at the moment, which can handle deltas pretty trivially (here's the relevant line in the JS IPC reader: https://github.com/apache/arrow/blob

Re: Narrowing down Arrow 0.9.0 backlog

2018-02-08 Thread Uwe L. Korn
Hello, I have assigned myself to all things that I would like to see in 0.9.0 and will try to implement all of them as soon as possible. Sadly, this is probably all I can do until the release besides the typical maintenance and code review. Greetings Uwe On Thu, Feb 8, 2018, at 1:17 AM, Wes Mc

[jira] [Created] (ARROW-2116) [JS] Implement IPC writer

2018-02-08 Thread Brian Hulette (JIRA)
Brian Hulette created ARROW-2116: Summary: [JS] Implement IPC writer Key: ARROW-2116 URL: https://issues.apache.org/jira/browse/ARROW-2116 Project: Apache Arrow Issue Type: Bug Comp

[jira] [Created] (ARROW-2115) [JS] Test arrow data production in integration test

2018-02-08 Thread Brian Hulette (JIRA)
Brian Hulette created ARROW-2115: Summary: [JS] Test arrow data production in integration test Key: ARROW-2115 URL: https://issues.apache.org/jira/browse/ARROW-2115 Project: Apache Arrow Issu

Re: [DISCUSS] Union Vector

2018-02-08 Thread Li Jin
Hi All, I'd like to bump this thread for more discussion. There is current a Java PR to make the current union type match the spec: https://github.com/apache/arrow/pull/987. Since there is a need for a "simple union" , i.e., a union that can only have one of each "minor type" and have fixed type

[jira] [Created] (ARROW-2114) [Python] Pull latest docker manylinux1 image

2018-02-08 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-2114: -- Summary: [Python] Pull latest docker manylinux1 image Key: ARROW-2114 URL: https://issues.apache.org/jira/browse/ARROW-2114 Project: Apache Arrow Issue Type: Tas

[jira] [Created] (ARROW-2113) [Python] Connect to hdfs failing with "pyarrow.lib.ArrowIOError: HDFS connection failed"

2018-02-08 Thread Michal Danko (JIRA)
Michal Danko created ARROW-2113: --- Summary: [Python] Connect to hdfs failing with "pyarrow.lib.ArrowIOError: HDFS connection failed" Key: ARROW-2113 URL: https://issues.apache.org/jira/browse/ARROW-2113