[jira] [Created] (ARROW-7999) [C++] Fix crash on corrupt Map array input (OSS-Fuzz)

2020-03-04 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-7999: - Summary: [C++] Fix crash on corrupt Map array input (OSS-Fuzz) Key: ARROW-7999 URL: https://issues.apache.org/jira/browse/ARROW-7999 Project: Apache Arrow

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-04 Thread Fan Liya
Hi Wes, I am thinking of adding an option named "USER_DEFINED" (or something similar) to enum CompressionType in your proposal. IMO, this option should be used primarily in Flight. Best, Liya Fan On Wed, Mar 4, 2020 at 11:12 AM Wes McKinney wrote: > On Tue, Mar 3, 2020, 8:11 PM Fan Liya wrote

Dedicated hardware for Arrow CI / benchmarking [was Re: CI setup on dedicated arm hardware]

2020-03-04 Thread Wes McKinney
hi folks, The tornado the night before last in Nashville, Tennessee temporarily disabled the physical hardware that I have been running there where we've been running "Ursabot" builds and where we've been experimenting with other self-hosted CI solutions like GitHub Actions Self-Hosted Runners and

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-04 Thread Wes McKinney
Okay, I guess my question is how the receiver is going to be able to determine how to "rehydrate" the record batch buffers: What I've proposed amounts to the following: * UNCOMPRESSED: the current behavior * ZSTD/LZ4/...: each buffer is compressed and written with an int64 length prefix (I'm clo

[ANNOUNCE] New Arrow PMC member: Neal Richardson

2020-03-04 Thread Wes McKinney
The Project Management Committee (PMC) for Apache Arrow has invited Neal Richardson to become a PMC member and we are pleased to announce that Neal has accepted. Congratulations and welcome!

[ANNOUNCE] New Arrow PMC member: Francois Saint-Jacques

2020-03-04 Thread Wes McKinney
The Project Management Committee (PMC) for Apache Arrow has invited Francois Saint-Jacques to become a PMC member and we are pleased to announce that Francois has accepted. Congratulations and welcome!

Timeline for next major Arrow release (0.17.0?), timed release planning

2020-03-04 Thread Wes McKinney
hi folks, Whether or not we complete the necessary integration testing to be able to make a 1.0 release, we will likely want to make a major release within 6 to 8 weeks from now, with the issue backlog being appropriate cleared out around Friday April 17 or so. Does this seem reasonable? Regardin

[jira] [Created] (ARROW-8000) [C++] gcc 4.8 build failures

2020-03-04 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8000: --- Summary: [C++] gcc 4.8 build failures Key: ARROW-8000 URL: https://issues.apache.org/jira/browse/ARROW-8000 Project: Apache Arrow Issue Type: Bug Com

[jira] [Created] (ARROW-8001) [C++][Dataset] R and Python bindings for dataset writing

2020-03-04 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-8001: -- Summary: [C++][Dataset] R and Python bindings for dataset writing Key: ARROW-8001 URL: https://issues.apache.org/jira/browse/ARROW-8001 Project: Apache Arrow

[jira] [Created] (ARROW-8002) [C++][Dataset] Dataset writing should let you (re)partition the data

2020-03-04 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-8002: -- Summary: [C++][Dataset] Dataset writing should let you (re)partition the data Key: ARROW-8002 URL: https://issues.apache.org/jira/browse/ARROW-8002 Project: Apach

[jira] [Created] (ARROW-8003) [C++] -DBZip2_SOURCE=BUNDLED fails when building with clang

2020-03-04 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8003: --- Summary: [C++] -DBZip2_SOURCE=BUNDLED fails when building with clang Key: ARROW-8003 URL: https://issues.apache.org/jira/browse/ARROW-8003 Project: Apache Arrow

[NIGHTLY] Arrow Build Report for Job nightly-2020-03-04-0

2020-03-04 Thread Crossbow
Arrow Build Report for Job nightly-2020-03-04-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-04-0 Failed Tasks: - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-04-0-travis-gandiva-jar-osx - gandiva-jar-t

Missing Windows 35 wheel for 0.16.0

2020-03-04 Thread Mark Keller
Hi, Thank you for all your work, I was trying to update `snowflake-connector-python` project to work with the newest `pyarrow` release 0.16.0. However; it seems like it’s missing the Windows Python 3.5 wheel file on PyPi. Was this on purpose? I found: https://issues.apache.org/jira/browse/ARROW-

Re: Missing Windows 35 wheel for 0.16.0

2020-03-04 Thread Wes McKinney
hi Mark -- looks like the 3.5 package was dropped due to problems with dependencies in https://github.com/apache/arrow/pull/5569 I'm sure we'd be fine uploading a 3.5 wheel if someone can provide a patch on top of the 0.16.0 branch to fix the package build on Python 3.5 We're considering droppin

[jira] [Created] (ARROW-8004) [Python] Define API for user-defined conversions of array cell values in pyarrow.array

2020-03-04 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8004: --- Summary: [Python] Define API for user-defined conversions of array cell values in pyarrow.array Key: ARROW-8004 URL: https://issues.apache.org/jira/browse/ARROW-8004 Pr

Status of Ursabot helper bot for Crossbow PR testing

2020-03-04 Thread Wes McKinney
Due to the tornado in Nashville in the early morning on Tuesday, ursabot is down with no ETA in sight for coming back up (my property was not damaged, but 600 electrical poles were taken down by the tornado and so power could be out for a few days or a few weeks, see https://twitter.com/nespower fo

[jira] [Created] (ARROW-8005) [Website] Review and remove any usages of Apache dist system from website / tools

2020-03-04 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8005: --- Summary: [Website] Review and remove any usages of Apache dist system from website / tools Key: ARROW-8005 URL: https://issues.apache.org/jira/browse/ARROW-8005 Project

[jira] [Created] (ARROW-8006) unsafe arrow dictionary recovered from parquet

2020-03-04 Thread Pierre Belzile (Jira)
Pierre Belzile created ARROW-8006: - Summary: unsafe arrow dictionary recovered from parquet Key: ARROW-8006 URL: https://issues.apache.org/jira/browse/ARROW-8006 Project: Apache Arrow Issue T