+1
On macOS 12.6 (arm64), Python 3.9.12, I ran:
TEST_DEFAULT=0 TEST_WHEELS=1 dev/release/verify-release-candidate.sh
10.0.0
and it succeeded, but some Python subprocess or something crashed in the
test suite--I think it was around test_pandas or test_plasma but I wasn't
paying close attention. The Apple crash log said, among other things:
Application Specific Information: abort() called
Kernel Triage: VM - pmap_enter failed with resource shortage
Given the "Release candidate looks good!" message at the end, I don't know
that this is critical or meaningful.
In addition to the wheels test, I also did the R packaging release prep
steps (autobrew, rtools-packages) and ran pre-submission checks for CRAN,
and we are ready to go there.
Neal
On Tue, Oct 25, 2022 at 12:21 PM Matt Topol <zotthewiz...@gmail.com> wrote:
> Looking at the current version of the `verify-release-candidate.sh` script,
> it looks like it does set GOROOT and GOPATH to be subdirs of
> $ARROW_TMPDIR/go. It installs go to ${ARROW_TMPDIR}/go etc. But it looks
> like it checks $USE_CONDA and if that is >0 it uses Conda to install Go but
> doesn't set GOPATH/GOROOT/etc. env vars and doesn't run `go install` at
> all.
>
> In the case where you're using Conda, `go install` would install the
> binaries to something under the conda_prefix. I'll open a PR to run `go
> install ./...` in test_go() and if you see anything wrong with the
> description above as far as the logic for USE_CONDA vs installing and using
> it entirely under ARROW_TMPDIR, please add on to the PR. I'll tag you in it
> Kou.
>
> That said, as before I still don't believe this should be a blocker at all
> for the v10.0.0 release.
>
> --Matt
>
> On Mon, Oct 24, 2022 at 11:08 PM Sutou Kouhei <k...@clear-code.com> wrote:
>
> > Hi,
> >
> > > What version of Go is your system Go?
> >
> > $ go version
> > go version go1.19 linux/amd64
> >
> > > Also, can you go to the `go` subdirectory of the repo and
> > > run `go install ./...` then try again?
> >
> > Yes. "go install ./..." created
> > ~/go/bin/arrow-flight-integration-{client,server}.
> >
> > Should we run "go install ./..." in test_go() in
> > verify-release-candidate.sh?
> >
> > BTW, can we set GOPATH to under $ARROW_TMPDIR such as
> > $ARROW_TMPDIR/go with system Go? I hope that
> > verify-release-candidate.sh doesn't output any files outside
> > $ARROW_TMPDIR.
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In <CAH4123YeLvjw3Zb2Ca0frkk2=nbg7l-arosxgj9yh13oidt...@mail.gmail.com>
> > "Re: [VOTE] Release Apache Arrow 10.0.0 - RC0" on Mon, 24 Oct 2022
> > 19:58:40 -0400,
> > Matt Topol <zotthewiz...@gmail.com> wrote:
> >
> > > @kou What version of Go is your system Go? Also, can you go to the `go`
> > > subdirectory of the repo and run `go install ./...` then try again?
> > Running
> > > that `go install ./...` command should build the binaries that were
> > > missing. Unless you have the environment variable `GOBIN` set, it will
> > > place the built binaries into `$(go env GOPATH)/bin`. It's possible
> that
> > > the issue is needing GOPATH to be set since the archery scripts default
> > to
> > > using $HOME/go if there's no GOPATH set.
> > >
> > > I agree that it shouldn't be a blocker, but I'll take a look at the
> > verify
> > > scripts to see what can be changed to improve this.
> > >
> > > --Matt
> > >
> > > On Mon, Oct 24, 2022 at 5:14 PM Sutou Kouhei <k...@clear-code.com>
> wrote:
> > >
> > >> +1
> > >>
> > >> I ran the followings on Debian GNU/Linux sid:
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_SOURCE=1 \
> > >> TEST_INTEGRATION_GO=0 \
> > >> LANG=C \
> > >> TZ=UTC \
> > >> CUDAToolkit_ROOT=/usr \
> > >> ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
> > >> -Dxsimd_SOURCE=BUNDLED" \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_APT=1 \
> > >> LANG=C \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_BINARY=1 \
> > >> LANG=C \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_JARS=1 \
> > >> LANG=C \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_WHEELS=1 \
> > >> LANG=C \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> * TEST_DEFAULT=0 \
> > >> TEST_YUM=1 \
> > >> LANG=C \
> > >> dev/release/verify-release-candidate.sh 10.0.0 0
> > >>
> > >> with:
> > >>
> > >> * .NET SDK (6.0.400)
> > >> * Python 3.10.6
> > >> * Python 3.9.13
> > >> * gcc (Debian 12.2.0-1) 12.2.0
> > >> * nvidia-cuda-dev 11.4.3-4+b1
> > >> * openjdk version "18.0.2-ea" 2022-07-19
> > >> * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
> > >>
> > >> Notes:
> > >>
> > >> Integration test was failed because Go's
> > >> arrow-flight-integration-client and
> > >> arrow-flight-integration-server weren't found.
> > >>
> > >> For example:
> > >>
> > >> ##########################################################
> > >> Flight: C++ serving, Go requesting
> > >> ##########################################################
> > >> ==========================================================
> > >> Testing file duplicate_fieldnames
> > >> ==========================================================
> > >> -- Skipping test because consumer Go does not support
> > >> ==========================================================
> > >> Testing file flight_sql:extension
> > >> ==========================================================
> > >> -- Skipping test because consumer Go does not support
> > >> ==========================================================
> > >> Testing file datetime
> > >> ==========================================================
> > >> Traceback (most recent call last):
> > >> File
> > >>
> >
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/runner.py",
> > >> line 376, in _run_flight_test_case
> > >> consumer.flight_request(port, **client_args)
> > >> File
> > >>
> >
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/tester_go.py",
> > >> line 121, in flight_request
> > >> run_cmd(cmd)
> > >> File
> > >>
> >
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/util.py",
> > >> line 133, in run_cmd
> > >> output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
> > >> File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
> > >> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
> > >> File "/usr/lib/python3.10/subprocess.py", line 501, in run
> > >> with Popen(*popenargs, **kwargs) as process:
> > >> File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
> > >> self._execute_child(args, executable, preexec_fn, close_fds,
> > >> File "/usr/lib/python3.10/subprocess.py", line 1845, in
> _execute_child
> > >> raise child_exception_type(errno_num, err_msg, err_filename)
> > >> FileNotFoundError: [Errno 2] No such file or directory:
> > >> '/home/kou/go/bin/arrow-flight-integration-client'
> > >>
> > >> I skipped integration test with Go with
> > >> TEST_INTEGRATION_GO=0. I don't think that this is a blocker
> > >> because it passed on GitHub Actions:
> > >>
> > >>
> >
> https://github.com/ursacomputing/crossbow/actions/runs/3293278249/jobs/5429607084#step:5:74963
> > >>
> > >> I think that this was caused because I used system Go. (The
> > >> above verification on GitHub Actions installs Go from
> > >> https://dl.google.com/go/ .) We may need to improve our
> > >> verification script for system Go. (Should we set GOPATH or
> > >> something?)
> > >>
> > >>
> > >> Thanks,
> > >> --
> > >> kou
> > >>
> > >>
> > >> In <20221021.150638.710084994470067293....@clear-code.com>
> > >> "[VOTE] Release Apache Arrow 10.0.0 - RC0" on Fri, 21 Oct 2022
> > 15:06:38
> > >> +0900 (JST),
> > >> Sutou Kouhei <k...@clear-code.com> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I would like to propose the following release candidate (RC0) of
> > Apache
> > >> > Arrow version 10.0.0. This is a release consisting of 470
> > >> > resolved JIRA issues[1].
> > >> >
> > >> > This release candidate is based on commit:
> > >> > 89f9a0948961f6e94f1ef5e4f310b707d22a3c11 [2]
> > >> >
> > >> > The source release rc0 is hosted at [3].
> > >> > The binary artifacts are hosted at [4][5][6][7][8][9][10][11].
> > >> > The changelog is located at [12].
> > >> >
> > >> > Please download, verify checksums and signatures, run the unit
> tests,
> > >> > and vote on the release. See [13] for how to validate a release
> > >> candidate.
> > >> >
> > >> > See also a verification result on GitHub pull request [14].
> > >> >
> > >> > The vote will be open for at least 72 hours.
> > >> >
> > >> > [ ] +1 Release this as Apache Arrow 10.0.0
> > >> > [ ] +0
> > >> > [ ] -1 Do not release this as Apache Arrow 10.0.0 because...
> > >> >
> > >> > [1]:
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%2010.0.0
> > >> > [2]:
> > >>
> >
> https://github.com/apache/arrow/tree/89f9a0948961f6e94f1ef5e4f310b707d22a3c11
> > >> > [3]:
> > >> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-10.0.0-rc0
> > >> > [4]: https://apache.jfrog.io/artifactory/arrow/almalinux-rc/
> > >> > [5]: https://apache.jfrog.io/artifactory/arrow/amazon-linux-rc/
> > >> > [6]: https://apache.jfrog.io/artifactory/arrow/centos-rc/
> > >> > [7]: https://apache.jfrog.io/artifactory/arrow/debian-rc/
> > >> > [8]: https://apache.jfrog.io/artifactory/arrow/java-rc/10.0.0-rc0
> > >> > [9]: https://apache.jfrog.io/artifactory/arrow/nuget-rc/10.0.0-rc0
> > >> > [10]:
> https://apache.jfrog.io/artifactory/arrow/python-rc/10.0.0-rc0
> > >> > [11]: https://apache.jfrog.io/artifactory/arrow/ubuntu-rc/
> > >> > [12]:
> > >>
> >
> https://github.com/apache/arrow/blob/89f9a0948961f6e94f1ef5e4f310b707d22a3c11/CHANGELOG.md
> > >> > [13]:
> > >>
> >
> https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates
> > >> > [14]: https://github.com/apache/arrow/pull/14466
> > >>
> >
>
