Re: Preparing for 0.17.0 Arrow release

2020-04-12 Thread Krisztián Szűcs
if you mean he connection error that's a flake. The kartothek build nightly passes in the last 3 reports. On Sun, Apr 12, 2020 at 3:57 AM Wes McKinney wrote: > > I'm not able to reproduce the test-conda-python-3.7-kartothek-master > failure locally with docker-compose, is that a flake or real? >

Re: Preparing for 0.17.0 Arrow release

2020-04-11 Thread Wes McKinney
I'm not able to reproduce the test-conda-python-3.7-kartothek-master failure locally with docker-compose, is that a flake or real? On Sat, Apr 11, 2020 at 7:29 AM Krisztián Szűcs wrote: > > On Sat, Apr 11, 2020 at 12:37 PM Antoine Pitrou wrote: > > > > > > Le 11/04/2020 à 12:34, Krisztián Szűcs

Re: Preparing for 0.17.0 Arrow release

2020-04-11 Thread Krisztián Szűcs
On Sat, Apr 11, 2020 at 12:37 PM Antoine Pitrou wrote: > > > Le 11/04/2020 à 12:34, Krisztián Szűcs a écrit : > > - test-conda-python-3.7-turbodbc-latest: > > - test-conda-python-3.7-turbodbc-master: > > The latest is important here, because the release would break the > > interoperability with

Re: Preparing for 0.17.0 Arrow release

2020-04-11 Thread Antoine Pitrou
Le 11/04/2020 à 12:34, Krisztián Szűcs a écrit : > - test-conda-python-3.7-turbodbc-latest: > - test-conda-python-3.7-turbodbc-master: > The latest is important here, because the release would break the > interoperability with turbodbc. > I need feedback on this from Uwe. For transparency, t

Re: Preparing for 0.17.0 Arrow release

2020-04-11 Thread Krisztián Szűcs
We'll receive three nightly reports per day until the release, see the failing tasks there. Failed Tasks: - conda-osx-*: - conda-win-*: - wheel-osx-cp38: - wheel-win-*: Python dataset tests are failing, possible related to silently ignoring paths when reading via the new ParquetDataset. Not tr

Re: Preparing for 0.17.0 Arrow release

2020-04-10 Thread Wes McKinney
None of the open issues look like they should prevent a release candidate from being cut. Fixing the nightly and packaging builds seems like the last remaining task, but some of them need to be tracked by JIRA issues. Can you write a list here of what's definitely currently broken? On Fri, Apr 10

Re: Preparing for 0.17.0 Arrow release

2020-04-10 Thread Krisztián Szűcs
We still have 12 open issues, about half of them are not essential. The more pressing problem is the number of failing builds including packaging builds (see the nightly reports). Releasing today doesn't look realistic, but if we're able to resolve the issues over the weekend I can start the relea

Re: Preparing for 0.17.0 Arrow release

2020-04-06 Thread Andy Grove
There are two trivial Rust PRs pending that I would like to see merged for the release. ARROW-7794: [Rust] Support releasing arrow-flight https://github.com/apache/arrow/pull/6858 ARROW-8357: [Rust] [DataFusion] Dockerfile for CLI is missing format dir https://github.com/apache/arrow/pull/6860

Re: Preparing for 0.17.0 Arrow release

2020-04-06 Thread Antoine Pitrou
Also nice to have perhaps (PR available and several back-and-forths already): * ARROW-7610: [Java] Finish support for 64 bit int allocations Needs a Java committer to decide... Regards Antoine. Le 06/04/2020 à 00:24, Wes McKinney a écrit : > We are getting close to the 0.17.0 endgame. > >

Re: Preparing for 0.17.0 Arrow release

2020-04-06 Thread Antoine Pitrou
Hi, I added the following issue to the cpp-1.6.0 milestone: * PARQUET-1835 [C++] Fix crashes on invalid input (OSS-Fuzz) There's a PR up for it and it's simple enough to be validated quickly, IMHO. Regards Antoine. Le 06/04/2020 à 00:24, Wes McKinney a écrit : > We are getting close to the

Re: Preparing for 0.17.0 Arrow release

2020-04-06 Thread Wes McKinney
That may be so. If we do partially revert it (the dict return value is the only thing probably that needs to be changed), we need to get the downstream libraries to make changes to allow us to make this change. Another option is returning the KV wrapper via another attribute. On Mon, Apr 6, 2020,

Re: Preparing for 0.17.0 Arrow release

2020-04-06 Thread Antoine Pitrou
Hmm, if downstream libraries were expecting a dict, perhaps we'll need to revert that change... Regards Antoine. Le 06/04/2020 à 08:50, Joris Van den Bossche a écrit : > We also have a recent regression related to the KeyValueMetadata wrapping > python that is causing failures in downstream l

Re: Preparing for 0.17.0 Arrow release

2020-04-05 Thread Joris Van den Bossche
We also have a recent regression related to the KeyValueMetadata wrapping python that is causing failures in downstream libraries, that seems a blocker for the release: https://issues.apache.org/jira/browse/ARROW-8342 On Mon, 6 Apr 2020 at 00:25, Wes McKinney wrote: > We are getting close to the

Re: Preparing for 0.17.0 Arrow release

2020-04-05 Thread Wes McKinney
We are getting close to the 0.17.0 endgame. Here are the 18 JIRAs still in the 0.17.0 milestone. There are a few issues without patches yet so we should decide quickly whether they need to be included. Are they any blocking issues not accounted for in the milestone? * ARROW-6947 [Rust] [DataFusio

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Fan Liya
I see ARROW-6871 in the list. It seems it has some bugs, which are being fixed by ARROW-8239. So I have added ARROW-8239 to the list. The PR for ARROW-8239 is already approved, so it is expected to be resolved soon. Best, Liya Fan On Wed, Apr 1, 2020 at 12:01 PM Micah Kornfield wrote: > I move

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Micah Kornfield
I moved the Java issues out of 0.17.0, they seem complex enough or not of enough significance to make them blockers for 0.17.0 release. If owners of the issues disagree please move them back int. On Tue, Mar 31, 2020 at 6:05 PM Wes McKinney wrote: > We've made good progress, but there are still

Re: Preparing for 0.17.0 Arrow release

2020-03-31 Thread Wes McKinney
We've made good progress, but there are still 35 issues in the backlog. Some of them are documentation related, but there are some functionality-related patches that could be at risk. If all could review again to trim out anything that isn't going to make the cut for 0.17.0, please do On Wed, Mar

Re: Preparing for 0.17.0 Arrow release

2020-03-25 Thread Andy Grove
I just took a first pass at reviewing the Java and Rust issues and removed some from the 0.17.0 release. There are a few small Rust issues that I am actively working on for this release. Thanks. On Wed, Mar 25, 2020 at 1:13 PM Wes McKinney wrote: > hi Neal, > > Thanks for helping coordinate. I

Re: Preparing for 0.17.0 Arrow release

2020-03-25 Thread Wes McKinney
hi Neal, Thanks for helping coordinate. I agree we should be in a position to release sometime next week. Can folks from the Rust and Java side review issues in the backlog? According to the dashboard there are 19 Rust issues open and 7 Java issues. Thanks On Tue, Mar 24, 2020 at 10:01 AM Neal