We'll receive three nightly reports per day until the release, see the failing tasks there.
Failed Tasks: - conda-osx-*: - conda-win-*: - wheel-osx-cp38: - wheel-win-*: Python dataset tests are failing, possible related to silently ignoring paths when reading via the new ParquetDataset. Not trivial to reproduce. - test-conda-python-3.7-hdfs-2.9.2: Unable to download hadoop from the apache mirror, probably not a blocker but would be nice to see that hdfs tests are passing. - test-conda-python-3.7-kartothek-master: - test-conda-python-3.7-turbodbc-latest: - test-conda-python-3.7-turbodbc-master: The latest is important here, because the release would break the interoperability with turbodbc. I need feedback on this from Uwe. On Fri, Apr 10, 2020 at 10:18 PM Wes McKinney <wesmck...@gmail.com> wrote: > > None of the open issues look like they should prevent a release > candidate from being cut. > > Fixing the nightly and packaging builds seems like the last remaining > task, but some of them need to be tracked by JIRA issues. Can you > write a list here of what's definitely currently broken? > > On Fri, Apr 10, 2020 at 1:35 PM Krisztián Szűcs > <szucs.kriszt...@gmail.com> wrote: > > > > We still have 12 open issues, about half of them are not essential. > > The more pressing problem is the number of failing builds including > > packaging builds (see the nightly reports). > > > > Releasing today doesn't look realistic, but if we're able to resolve > > the issues over the weekend I can start the release procedure on > > Monday. > > > > Thanks, Krisztian > > > > On Tue, Apr 7, 2020 at 4:31 AM Andy Grove <andygrov...@gmail.com> wrote: > > > > > > There are two trivial Rust PRs pending that I would like to see merged for > > > the release. > > > > > > ARROW-7794: [Rust] Support releasing arrow-flight > > > > > > https://github.com/apache/arrow/pull/6858 > > > > > > ARROW-8357: [Rust] [DataFusion] Dockerfile for CLI is missing format dir > > > > > > https://github.com/apache/arrow/pull/6860 > > > > > > Thanks, > > > > > > Andy. > > > > > > > > > On Mon, Apr 6, 2020 at 6:55 AM Antoine Pitrou <anto...@python.org> wrote: > > > > > > > > > > > Also nice to have perhaps (PR available and several back-and-forths > > > > already): > > > > > > > > * ARROW-7610: [Java] Finish support for 64 bit int allocations > > > > > > > > Needs a Java committer to decide... > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > > > > Le 06/04/2020 à 00:24, Wes McKinney a écrit : > > > > > We are getting close to the 0.17.0 endgame. > > > > > > > > > > Here are the 18 JIRAs still in the 0.17.0 milestone. There are a few > > > > > issues without patches yet so we should decide quickly whether they > > > > > need to be included. Are they any blocking issues not accounted for in > > > > > the milestone? > > > > > > > > > > * ARROW-6947 [Rust] [DataFusion] Add support for scalar UDFs > > > > > > > > > > Patch available > > > > > > > > > > * ARROW-7794 [Rust] cargo publish fails for arrow-flight due to > > > > > relative path to Flight.proto > > > > > > > > > > No patch yet > > > > > > > > > > * ARROW-7222 [Python][Release] Wipe any existing generated Python API > > > > > documentation when updating website > > > > > > > > > > This issue needs to be addressed by the release manager and the > > > > > Confluence instructions must be updated. > > > > > > > > > > * ARROW-7891 [C++] RecordBatch->Equals should also have a > > > > > check_metadata argument > > > > > > > > > > Patch available that needs to be reviewed and approved > > > > > > > > > > * ARROW-8164: [C++][Dataset] Let datasets be viewable with > > > > > non-identical > > > > schema > > > > > > > > > > Patch available, but failures to be resolved > > > > > > > > > > * ARROW-7965: [Python] Hold a reference to the dataset factory for > > > > > later > > > > reuse > > > > > > > > > > Depends on ARROW-8164, will require rebase > > > > > > > > > > * ARROW-8039: [Python][Dataset] Support using dataset API in > > > > > pyarrow.parquet with a minimal ParquetDataset shim > > > > > > > > > > Patch pending > > > > > > > > > > * ARROW-8047: [Python][Documentation] Document migration from > > > > > ParquetDataset to pyarrow.datasets > > > > > > > > > > May be tackled beyond 0.17.0 > > > > > > > > > > * ARROW-8063: [Python] Add user guide documentation for Datasets API > > > > > > > > > > May be tackled beyond 0.17.0 > > > > > > > > > > * ARROW-8149 [C++/Python] Enable CUDA Support in conda recipes > > > > > > > > > > Does not seem strictly necessary for release, since a packaging issue > > > > > > > > > > * ARROW-8162: [Format][Python] Add serialization for CSF sparse > > > > > tensors > > > > > > > > > > Patch available, but needs review. May > > > > > > > > > > * ARROW-8213: [Python][Dataset] Opening a dataset with a local > > > > > incorrect path gives confusing error message > > > > > > > > > > Nice to have, but not essential > > > > > > > > > > * ARROW-8266: [C++] Add backup mirrors for external project source > > > > downloads > > > > > > > > > > Patch available, nice to have > > > > > > > > > > * ARROW-8275 [Python][Docs] Review Feather + IPC file documentation > > > > > per "Feather V2" changes > > > > > > > > > > Patch available > > > > > > > > > > * ARROW-8300 [R] Documentation and changelog updates for 0.17 > > > > > > > > > > Patch available > > > > > > > > > > * ARROW-8320 [Documentation][Format] Clarify (lack of) alignment > > > > > requirements in C data interface > > > > > > > > > > Patch available > > > > > > > > > > * ARROW-8330: [Documentation] The post release script generates the > > > > > documentation with a development version > > > > > > > > > > Patch available > > > > > > > > > > * ARROW-8335: [Release] Add crossbow jobs to run release verification > > > > > > > > > > Patch in progress > > > > > > > > > > On Tue, Mar 31, 2020 at 11:23 PM Fan Liya <liya.fa...@gmail.com> > > > > > wrote: > > > > >> > > > > >> I see ARROW-6871 in the list. > > > > >> It seems it has some bugs, which are being fixed by ARROW-8239. > > > > >> So I have added ARROW-8239 to the list. > > > > >> > > > > >> The PR for ARROW-8239 is already approved, so it is expected to be > > > > resolved > > > > >> soon. > > > > >> > > > > >> Best, > > > > >> Liya Fan > > > > >> > > > > >> On Wed, Apr 1, 2020 at 12:01 PM Micah Kornfield > > > > >> <emkornfi...@gmail.com> > > > > >> wrote: > > > > >> > > > > >>> I moved the Java issues out of 0.17.0, they seem complex enough or > > > > >>> not > > > > of > > > > >>> enough significance to make them blockers for 0.17.0 release. If > > > > owners of > > > > >>> the issues disagree please move them back int. > > > > >>> > > > > >>> On Tue, Mar 31, 2020 at 6:05 PM Wes McKinney <wesmck...@gmail.com> > > > > wrote: > > > > >>> > > > > >>>> We've made good progress, but there are still 35 issues in the > > > > >>>> backlog. Some of them are documentation related, but there are some > > > > >>>> functionality-related patches that could be at risk. If all could > > > > >>>> review again to trim out anything that isn't going to make the cut > > > > >>>> for > > > > >>>> 0.17.0, please do > > > > >>>> > > > > >>>> On Wed, Mar 25, 2020 at 2:39 PM Andy Grove <andygrov...@gmail.com> > > > > >>> wrote: > > > > >>>>> > > > > >>>>> I just took a first pass at reviewing the Java and Rust issues and > > > > >>>> removed > > > > >>>>> some from the 0.17.0 release. There are a few small Rust issues > > > > >>>>> that > > > > I > > > > >>> am > > > > >>>>> actively working on for this release. > > > > >>>>> > > > > >>>>> Thanks. > > > > >>>>> > > > > >>>>> > > > > >>>>> On Wed, Mar 25, 2020 at 1:13 PM Wes McKinney <wesmck...@gmail.com> > > > > >>>> wrote: > > > > >>>>> > > > > >>>>>> hi Neal, > > > > >>>>>> > > > > >>>>>> Thanks for helping coordinate. I agree we should be in a > > > > >>>>>> position to > > > > >>>>>> release sometime next week. > > > > >>>>>> > > > > >>>>>> Can folks from the Rust and Java side review issues in the > > > > >>>>>> backlog? > > > > >>>>>> According to the dashboard there are 19 Rust issues open and 7 > > > > >>>>>> Java > > > > >>>>>> issues. > > > > >>>>>> > > > > >>>>>> Thanks > > > > >>>>>> > > > > >>>>>> On Tue, Mar 24, 2020 at 10:01 AM Neal Richardson > > > > >>>>>> <neal.p.richard...@gmail.com> wrote: > > > > >>>>>>> > > > > >>>>>>> Hi all, > > > > >>>>>>> A few weeks ago, there seemed to be consensus (lazy, at least) > > > > >>>>>>> for > > > > >>> a > > > > >>>> 0.17 > > > > >>>>>>> release at the end of the month. Judging from > > > > >>>>>>> > > > > >>>> > > > > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.17.0+Release, > > > > >>>>>> it > > > > >>>>>>> looks like we're getting closer. > > > > >>>>>>> > > > > >>>>>>> I'd encourage everyone to review their backlogs and (1) bump > > > > >>>>>>> from > > > > >>>> 0.17 > > > > >>>>>>> scope any tickets they don't plan to finish this week, and (2) > > > > >>>>>>> if > > > > >>>> there > > > > >>>>>> are > > > > >>>>>>> any issues that should block release, make sure they are > > > > >>>>>>> flagged as > > > > >>>>>>> "blockers". > > > > >>>>>>> > > > > >>>>>>> Neal > > > > >>>>>>> > > > > >>>>>>> On Tue, Mar 10, 2020 at 7:39 AM Wes McKinney > > > > >>>>>>> <wesmck...@gmail.com> > > > > >>>>>> wrote: > > > > >>>>>>> > > > > >>>>>>>> It seems like the consensus is to push for a 0.17.0 major > > > > >>>>>>>> release > > > > >>>>>>>> sooner rather than doing a patch release, since releases in > > > > >>> general > > > > >>>>>>>> are costly. This is fine with me. I see that a 0.17.0 milestone > > > > >>> has > > > > >>>>>>>> been created in JIRA and some JIRA gardening has begun. Do you > > > > >>>> think > > > > >>>>>>>> we can be in a position to release by the week of March 23 or > > > > >>>>>>>> the > > > > >>>> week > > > > >>>>>>>> of March 30? > > > > >>>>>>>> > > > > >>>>>>>> On Thu, Mar 5, 2020 at 8:39 PM Wes McKinney > > > > >>>>>>>> <wesmck...@gmail.com > > > > >>>> > > > > >>>>>> wrote: > > > > >>>>>>>>> > > > > >>>>>>>>> If people are generally on board with accelerating a 0.17.0 > > > > >>> major > > > > >>>>>>>>> release, then I would suggest renaming "1.0.0" to "0.17.0" and > > > > >>>>>>>>> beginning to do issue gardening to whittle things down to > > > > >>>>>>>>> critical-looking bugs and high probability patches for the > > > > >>>>>>>>> next > > > > >>>>>> couple > > > > >>>>>>>>> of weeks. > > > > >>>>>>>>> > > > > >>>>>>>>> On Thu, Mar 5, 2020 at 11:31 AM Wes McKinney < > > > > >>>> wesmck...@gmail.com> > > > > >>>>>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>>> I recall there are some other issues that have been reported > > > > >>> or > > > > >>>>>> fixed > > > > >>>>>>>>>> that are critical and not yet marked with 0.16.1. > > > > >>>>>>>>>> > > > > >>>>>>>>>> I'm also OK with doing a 0.17.0 release sooner > > > > >>>>>>>>>> > > > > >>>>>>>>>> On Thu, Mar 5, 2020 at 11:31 AM Neal Richardson > > > > >>>>>>>>>> <neal.p.richard...@gmail.com> wrote: > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> I would also be more supportive of doing 0.17 earlier > > > > >>>> instead of > > > > >>>>>> a > > > > >>>>>>>> patch > > > > >>>>>>>>>>> release. > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> Neal > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> On Thu, Mar 5, 2020 at 9:29 AM Neal Richardson < > > > > >>>>>>>> neal.p.richard...@gmail.com> > > > > >>>>>>>>>>> wrote: > > > > >>>>>>>>>>> > > > > >>>>>>>>>>>> If releases were costless to make, I'd be all for it, but > > > > >>>> it's > > > > >>>>>> not > > > > >>>>>>>> clear > > > > >>>>>>>>>>>> to me that it's worth the diversion from other priorities > > > > >>>> to > > > > >>>>>> make > > > > >>>>>>>> a release > > > > >>>>>>>>>>>> right now. Nothing on > > > > >>>>>>>>>>>> > > > > >>>>>>>> > > > > >>>>>> > > > > >>>> > > > > >>> > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20%3D%20Resolved%20AND%20fixVersion%20%3D%200.16.1 > > > > >>>>>>>>>>>> jumps out to me as super urgent--what are you seeing as > > > > >>>>>> critical? > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> If we did decide to go forward, would it be possible to > > > > >>> do > > > > >>>> a > > > > >>>>>>>> release that > > > > >>>>>>>>>>>> is limited to the affected implementations (say, do a > > > > >>>>>> Python-only > > > > >>>>>>>> release)? > > > > >>>>>>>>>>>> That might reduce the cost of building and verifying > > > > >>>> enough to > > > > >>>>>>>> make it > > > > >>>>>>>>>>>> reasonable to consider. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> Neal > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> On Thu, Mar 5, 2020 at 8:19 AM Krisztián Szűcs < > > > > >>>>>>>> szucs.kriszt...@gmail.com> > > > > >>>>>>>>>>>> wrote: > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>> On Thu, Mar 5, 2020 at 5:07 PM Wes McKinney < > > > > >>>>>> wesmck...@gmail.com> > > > > >>>>>>>> wrote: > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> hi folks, > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> There have been a number of critical issues reported > > > > >>>> (many > > > > >>>>>> of > > > > >>>>>>>> them > > > > >>>>>>>>>>>>>> fixed already) since 0.16.0 was released. Is there > > > > >>>> interest > > > > >>>>>> in > > > > >>>>>>>>>>>>>> preparing a patch 0.16.1 release (with backported > > > > >>>> patches > > > > >>>>>> onto a > > > > >>>>>>>>>>>>>> maint-0.16.x branch as with 0.15.1) since the next > > > > >>> major > > > > >>>>>>>> release is a > > > > >>>>>>>>>>>>>> minimum of 6-8 weeks away from general availability? > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Did the 0.15.1 patch release helper script that > > > > >>>> Krisztian > > > > >>>>>> wrote > > > > >>>>>>>> get > > > > >>>>>>>>>>>>>> contributed as a PR? > > > > >>>>>>>>>>>>> Not yet, but it is available at > > > > >>>>>>>>>>>>> > > > > >>>>>> https://gist.github.com/kszucs/b2743546044ccd3215e5bb34fa0d76a0 > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Thanks > > > > >>>>>>>>>>>>>> Wes > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>> > > > > >>>>>>>> > > > > >>>>>> > > > > >>>> > > > > >>> > > > >
