[jira] [Created] (ARROW-8303) [Python] Fix test failure caused by non-deterministic dict key ordering on Python 3.5

2020-03-31 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8303: --- Summary: [Python] Fix test failure caused by non-deterministic dict key ordering on Python 3.5 Key: ARROW-8303 URL: https://issues.apache.org/jira/browse/ARROW-8303 Pro

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Fan Liya
I see ARROW-6871 in the list. It seems it has some bugs, which are being fixed by ARROW-8239. So I have added ARROW-8239 to the list. The PR for ARROW-8239 is already approved, so it is expected to be resolved soon. Best, Liya Fan On Wed, Apr 1, 2020 at 12:01 PM Micah Kornfield wrote: > I move

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Micah Kornfield
I moved the Java issues out of 0.17.0, they seem complex enough or not of enough significance to make them blockers for 0.17.0 release. If owners of the issues disagree please move them back int. On Tue, Mar 31, 2020 at 6:05 PM Wes McKinney wrote: > We've made good progress, but there are still

Re: The future of Parquet development for Arrow Rust?

2020-03-31 Thread Micah Kornfield
At least for testing, would using the new C data interface for FFI from Rust to C++ (where Rust code provides Arrow Data and a file path to write to?) be an easy to use short term solution? On Tue, Mar 31, 2020 at 7:42 AM Andy Grove wrote: > To get the ball rolling, here is a quick and dirty PR

Arrow sync call March 4 at 12:00 US/Eastern, 16:00 UTC

2020-03-31 Thread Neal Richardson
Hi all, Reminder that our biweekly call is coming up tomorrow/later today at https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes will be sent out to the mailing list afterward. Neal

[NIGHTLY] Arrow Build Report for Job nightly-2020-03-31-0

2020-03-31 Thread Crossbow
Arrow Build Report for Job nightly-2020-03-31-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-31-0 Failed Tasks: - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-31-0-azure-conda-linux-gcc-py36 - cond

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Wes McKinney
We've made good progress, but there are still 35 issues in the backlog. Some of them are documentation related, but there are some functionality-related patches that could be at risk. If all could review again to trim out anything that isn't going to make the cut for 0.17.0, please do On Wed, Mar

[jira] [Created] (ARROW-8302) Start plasma store with STDOUT, STDERR arguments

2020-03-31 Thread Tal Pritzker (Jira)
Tal Pritzker created ARROW-8302: --- Summary: Start plasma store with STDOUT, STDERR arguments Key: ARROW-8302 URL: https://issues.apache.org/jira/browse/ARROW-8302 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-8301) [C++][Python][R] Handle ChunkedArray and Table in C data interface

2020-03-31 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-8301: -- Summary: [C++][Python][R] Handle ChunkedArray and Table in C data interface Key: ARROW-8301 URL: https://issues.apache.org/jira/browse/ARROW-8301 Project: Apache

[jira] [Created] (ARROW-8300) [R] Documentation and changelog updates for 0.17

2020-03-31 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-8300: -- Summary: [R] Documentation and changelog updates for 0.17 Key: ARROW-8300 URL: https://issues.apache.org/jira/browse/ARROW-8300 Project: Apache Arrow Iss

[jira] [Created] (ARROW-8299) [C++] Reusable "optional ParallelFor" function for optional use of multithreading

2020-03-31 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8299: --- Summary: [C++] Reusable "optional ParallelFor" function for optional use of multithreading Key: ARROW-8299 URL: https://issues.apache.org/jira/browse/ARROW-8299 Project

[jira] [Created] (ARROW-8298) [C++][CI] MinGW builds fail building grpc

2020-03-31 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-8298: - Summary: [C++][CI] MinGW builds fail building grpc Key: ARROW-8298 URL: https://issues.apache.org/jira/browse/ARROW-8298 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-8297) [FlightRPC][C++] Implement Flight DoExchange for C++

2020-03-31 Thread David Li (Jira)
David Li created ARROW-8297: --- Summary: [FlightRPC][C++] Implement Flight DoExchange for C++ Key: ARROW-8297 URL: https://issues.apache.org/jira/browse/ARROW-8297 Project: Apache Arrow Issue Type: N

Re: [RESULT] [VOTE] Accept "DoExchange" RPC to Arrow Flight protocol

2020-03-31 Thread David Li
Thanks everyone! I will be updating the C++ draft PR soon and then following up with Python and Java. Best, David On 3/31/20, Wes McKinney wrote: > The vote carries with 3 binding +1 votes. Thanks all > > On Sun, Mar 29, 2020 at 10:26 PM Andy Grove wrote: >> >> +1 (binding) >> >> On Sat, Mar 2

[jira] [Created] (ARROW-8296) [C++][Dataset] IpcFileFormat should support writing files with compressed buffers

2020-03-31 Thread Ben Kietzman (Jira)
Ben Kietzman created ARROW-8296: --- Summary: [C++][Dataset] IpcFileFormat should support writing files with compressed buffers Key: ARROW-8296 URL: https://issues.apache.org/jira/browse/ARROW-8296 Project

[jira] [Created] (ARROW-8295) [C++][Dataset] IpcFileFormat should expliclity push down column projection

2020-03-31 Thread Ben Kietzman (Jira)
Ben Kietzman created ARROW-8295: --- Summary: [C++][Dataset] IpcFileFormat should expliclity push down column projection Key: ARROW-8295 URL: https://issues.apache.org/jira/browse/ARROW-8295 Project: Apach

[jira] [Created] (ARROW-8294) [Format][Flight] Add DoExchange RPC to Flight protocol

2020-03-31 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8294: --- Summary: [Format][Flight] Add DoExchange RPC to Flight protocol Key: ARROW-8294 URL: https://issues.apache.org/jira/browse/ARROW-8294 Project: Apache Arrow Iss

[RESULT] [VOTE] Accept "DoExchange" RPC to Arrow Flight protocol

2020-03-31 Thread Wes McKinney
The vote carries with 3 binding +1 votes. Thanks all On Sun, Mar 29, 2020 at 10:26 PM Andy Grove wrote: > > +1 (binding) > > On Sat, Mar 28, 2020 at 5:08 AM Antoine Pitrou wrote: > > > > > +1 (binding) > > > > > > Le 28/03/2020 à 01:44, Wes McKinney a écrit : > > > Hello, > > > > > > David M Li

[jira] [Created] (ARROW-8293) [Python] Run flake8 on python/examples also

2020-03-31 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8293: --- Summary: [Python] Run flake8 on python/examples also Key: ARROW-8293 URL: https://issues.apache.org/jira/browse/ARROW-8293 Project: Apache Arrow Issue Type: Im

[jira] [Created] (ARROW-8292) [Python][Dataset] Passthrough schema to Factory.finish() in dataset() function

2020-03-31 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-8292: Summary: [Python][Dataset] Passthrough schema to Factory.finish() in dataset() function Key: ARROW-8292 URL: https://issues.apache.org/jira/browse/ARROW-8292

[jira] [Created] (ARROW-8291) [Packaging] Conda nightly builds can't locate Numpy

2020-03-31 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-8291: -- Summary: [Packaging] Conda nightly builds can't locate Numpy Key: ARROW-8291 URL: https://issues.apache.org/jira/browse/ARROW-8291 Project: Apache Arrow

[jira] [Created] (ARROW-8290) [Python][Dataset] Improve ergonomy of the FileSystemDataset constructor

2020-03-31 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-8290: Summary: [Python][Dataset] Improve ergonomy of the FileSystemDataset constructor Key: ARROW-8290 URL: https://issues.apache.org/jira/browse/ARROW-8290

Re: The future of Parquet development for Arrow Rust?

2020-03-31 Thread Andy Grove
To get the ball rolling, here is a quick and dirty PR adding a test that writes an Arrow batch to a Parquet file. https://github.com/apache/arrow/pull/6785 I'll keep iterating on this but will gladly accept help or hand this off to someone better qualified. On Tue, Mar 31, 2020 at 8:15 AM Wes

[jira] [Created] (ARROW-8289) Implement Arrow Parquet writer

2020-03-31 Thread Andy Grove (Jira)
Andy Grove created ARROW-8289: - Summary: Implement Arrow Parquet writer Key: ARROW-8289 URL: https://issues.apache.org/jira/browse/ARROW-8289 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-8288) [Python] Expose with_ modifiers on DataType

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8288: --- Summary: [Python] Expose with_ modifiers on DataType Key: ARROW-8288 URL: https://issues.apache.org/jira/browse/ARROW-8288 Project: Apache Arrow Issue Type: Improvemen

Re: The future of Parquet development for Arrow Rust?

2020-03-31 Thread Wes McKinney
Here was the last discussion about this 6 months ago https://github.com/apache/parquet-testing/pull/9 I saw another PR come through like this so that's why I'm bringing it up again https://github.com/apache/parquet-testing/pull/11 On Tue, Mar 31, 2020 at 9:08 AM Andy Grove wrote: > > Hi Wes, >

[jira] [Created] (ARROW-8287) [Rust] Arrow examples should use utility to print results

2020-03-31 Thread Andy Grove (Jira)
Andy Grove created ARROW-8287: - Summary: [Rust] Arrow examples should use utility to print results Key: ARROW-8287 URL: https://issues.apache.org/jira/browse/ARROW-8287 Project: Apache Arrow Issu

Re: The future of Parquet development for Arrow Rust?

2020-03-31 Thread Andy Grove
Hi Wes, I agree that this is important. I have been looking at the Parquet implementation this morning and I do see code for writing files., along with roundtrip tests As you said, It isn't writing from Arrow types yet but I would hope that this would be relatively simple to add. I don't know how

[jira] [Created] (ARROW-8286) [Python] Creating dataset from pathlib results in UnionDataset instead of FileSystemDataset

2020-03-31 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-8286: Summary: [Python] Creating dataset from pathlib results in UnionDataset instead of FileSystemDataset Key: ARROW-8286 URL: https://issues.apache.org/jira/browse/ARR

[jira] [Created] (ARROW-8285) [Python][Dataset] ScalarExpression doesn't accept numpy scalars

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8285: --- Summary: [Python][Dataset] ScalarExpression doesn't accept numpy scalars Key: ARROW-8285 URL: https://issues.apache.org/jira/browse/ARROW-8285 Project: Apache Arrow I

[jira] [Created] (ARROW-8284) [C++][Dataset] Schema evolution for timestamp columns

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8284: --- Summary: [C++][Dataset] Schema evolution for timestamp columns Key: ARROW-8284 URL: https://issues.apache.org/jira/browse/ARROW-8284 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-8283) [C++/Python][Dataset] Non-existent files are silently dropped in pa.dataset.FileSystemDataset

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8283: --- Summary: [C++/Python][Dataset] Non-existent files are silently dropped in pa.dataset.FileSystemDataset Key: ARROW-8283 URL: https://issues.apache.org/jira/browse/ARROW-8283 Pro

[jira] [Created] (ARROW-8282) [C++/Python][Dataset] Support schema evolution for integer columns

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8282: --- Summary: [C++/Python][Dataset] Support schema evolution for integer columns Key: ARROW-8282 URL: https://issues.apache.org/jira/browse/ARROW-8282 Project: Apache Arrow

[jira] [Created] (ARROW-8281) [R] Name collision of arrow.dll on Windows

2020-03-31 Thread Uwe Korn (Jira)
Uwe Korn created ARROW-8281: --- Summary: [R] Name collision of arrow.dll on Windows Key: ARROW-8281 URL: https://issues.apache.org/jira/browse/ARROW-8281 Project: Apache Arrow Issue Type: Improvement