Re: How to get "standard" binary columns out of a pyarrow table

2018-01-31 Thread Eli
Hey Wes, What I meant by "standard" is the binary representation of a specific type aggregated together. The int32 column [1,2,3] would make '\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00' for example. This is already available via Python's struct.pack(), array.array().tostring() or np.a

Re: How to get "standard" binary columns out of a pyarrow table

2018-01-31 Thread Wes McKinney
hi Eli, This isn't available at the moment, but one could make the internal buffers in an array accessible in Python. How would you handle nulls in this scenario (the bytes for a null value in a primitive array can be any value)? How would one handle things other than numbers? - Wes On Wed, Jan

[jira] [Created] (ARROW-2066) [Python] Document reading Parquet files from Azure Blob Store

2018-01-31 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2066: --- Summary: [Python] Document reading Parquet files from Azure Blob Store Key: ARROW-2066 URL: https://issues.apache.org/jira/browse/ARROW-2066 Project: Apache Arrow

[jira] [Created] (ARROW-2067) "pip install" doesn't work from source tree

2018-01-31 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-2067: - Summary: "pip install" doesn't work from source tree Key: ARROW-2067 URL: https://issues.apache.org/jira/browse/ARROW-2067 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2068) [Python] Expose Array's buffers to Python users

2018-01-31 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2068: --- Summary: [Python] Expose Array's buffers to Python users Key: ARROW-2068 URL: https://issues.apache.org/jira/browse/ARROW-2068 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2069) [Python] Document that Plasma is not (yet) supported on Windows

2018-01-31 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2069: --- Summary: [Python] Document that Plasma is not (yet) supported on Windows Key: ARROW-2069 URL: https://issues.apache.org/jira/browse/ARROW-2069 Project: Apache Arrow

[jira] [Created] (ARROW-2070) [Python] chdir logic in setup.py buggy

2018-01-31 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-2070: - Summary: [Python] chdir logic in setup.py buggy Key: ARROW-2070 URL: https://issues.apache.org/jira/browse/ARROW-2070 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-2071) [Python] Reduce runtime of builds in Travis CI

2018-01-31 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2071: --- Summary: [Python] Reduce runtime of builds in Travis CI Key: ARROW-2071 URL: https://issues.apache.org/jira/browse/ARROW-2071 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-2072) [Python] decimal128.byte_width crashes

2018-01-31 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-2072: - Summary: [Python] decimal128.byte_width crashes Key: ARROW-2072 URL: https://issues.apache.org/jira/browse/ARROW-2072 Project: Apache Arrow Issue Type: Bug