The place where the segfault is triggered in the R nightlies is a couple of tests after the one I added in that patch. If that patch is causing the segfaults, we can skip the new test ( https://github.com/apache/arrow/blob/master/r/tests/testthat/test-parquet.R#L125) and investigate later. The patch is exercising previously existing codepaths that were not tested, so I don't think that identifying and fixing the segfault should be release blocking (though we should clearly fix it).
Neal On Wed, Jan 29, 2020 at 7:33 AM David Li <li.david...@gmail.com> wrote: > The Flight leak should be unrelated to that commit, the failing test > already existed before - it's a flaky test > https://issues.apache.org/jira/browse/ARROW-7721. > > I'm hoping to look at the issue this week but we might just want to > disable the test for now. > > David > > On 1/29/20, Krisztián Szűcs <szucs.kriszt...@gmail.com> wrote: > > Hi, > > > > - The fuzzit builds has been disabled by Neal on the current master. > > - Created a PR to resolve occasionally failing python dataset tests [1] > > - Merged the fix for C# timezone error [2] > > - Merged various fixes for the release scripts. > > - The nightly Gandiva OS X build is still failing, but because of a > travis > > deployment timeout, which shouldn't block the release. > > > > We still have failing tests: > > - failing Spark integration test > > - failing nightly R builds (see the 2020-01-29 nightly report) > > - master reports a Java flight memory leak > > > > Spark: > > Joris created a fix for the immediate issue [3], but now we have a > > different > > spark test error, see the discussion in the PR [3]. > > I put up a PR [6] to check the regressions 0.16 arrow release would > > introduce > > for spark interoperability, and it turns out that arrow 0.15.1 is not > > compatible > > with neither spark 2.4.4 nor spark 2.4.5-rc1, so 0.16 arrow release could > > only be compatible with spark 3.0 or spark master which we have tests > for. > > So I'm a bit confused how to interpret arrow backward compatibility with > > Spark, thus what should and what should not block the release. > > Either way we'll need to fix the remaining spark issues and add nightly > > spark > > integration tests for both the next spark release and spark master. > > > > R: > > There is the same segfault in each R nightly builds [4]. There was a > single > > change [5] which could introduce the regression compared to the previous > > builds. > > I've tried to reproduce the builds using docker-compose, but locally > > 3.6-bionic > > has passed for me. I'm trying to wipe my local cache and rerun to see > > whether I can reproduce it. > > > > Java/Flight leak: > > The current master reports memory leak [6] which I guess is surfaced by > > change [7] > > > > If we manage to fix the issues above today than I can cut RC2 tomorrow. > > > > Thanks, Krisztian > > > > [1]: https://github.com/apache/arrow/pull/6319 > > [2]: https://github.com/apache/arrow/pull/6309 > > [3]: https://github.com/apache/arrow/pull/6312 > > [4]: https://github.com/ursa-labs/crossbow/branches/all?query=r-base > > [5]: > > > https://github.com/apache/arrow/commit/8b7911b086d120359e2000fbedb0c38c0f13f683 > > [6]: https://github.com/apache/arrow/runs/415037585#step:5:1533 > > [7]: > > > https://github.com/apache/arrow/commit/8b42288f58caa84a40bb7a13c1731ff919c934f2 > > > > On Wed, Jan 29, 2020 at 11:06 AM Sutou Kouhei <k...@clear-code.com> > wrote: > >> > >> Hi, > >> > >> > Thank you. After the C# download fix, I have the following C# test > >> > failure: > >> > https://gist.github.com/pitrou/d82ed1ff80db43b63f0c3d5e5f2474a4 > >> > >> https://github.com/apache/arrow/pull/6309 > >> will fix it. > >> > >> I think that this is a test problem, not an implementation > >> problem. > >> > >> > >> Workaround: > >> > >> TZ=UTC dev/release/verify-release-candidate.sh ... > >> > >> > >> Thanks, > >> -- > >> kou > >> > >> In <2bd07a17-600b-7f49-3ea1-a0b1acc91...@python.org> > >> "Re: [VOTE] Release Apache Arrow 0.16.0 - RC1" on Wed, 29 Jan 2020 > >> 10:11:46 +0100, > >> Antoine Pitrou <anto...@python.org> wrote: > >> > >> > > >> > Thank you. After the C# download fix, I have the following C# test > >> > failure: > >> > https://gist.github.com/pitrou/d82ed1ff80db43b63f0c3d5e5f2474a4 > >> > > >> > Regards > >> > > >> > Antoine. > >> > > >> > > >> > > >> > Le 29/01/2020 à 00:42, Sutou Kouhei a écrit : > >> >> Hi, > >> >> > >> >>> Source verification succeeded on Java and C++ and then failed > >> >>> downloading some C# thing: > >> >>> https://gist.github.com/pitrou/5c4a98387153ef415ef64b8aa2457e63 > >> >> > >> >> I've created a pull request to fix it: > >> >> https://github.com/apache/arrow/pull/6307 > >> >> > >> >> > >> >> Thanks, > >> >> -- > >> >> kou > > >
