In principle these aren't dealbreakers. Since the project has been
growing a lot lately, I'm not surprised to see some rough edges.
* After the release we should probably downgrade gbenchmark and report
the issue upstream
* The Plasma failure is definitely odd, I haven't seen any such thing
in Travis CI. Could it be related to the macOS version?
I got a bit further on Ubuntu 14.04 with CMake 3.12.0 from a ppa, but
failed on a JS integration test
==========================================================
Testing file
/tmp/arrow-0.11.0.E5KVJ/apache-arrow-0.11.0/integration/data/struct_example.json
==========================================================
-- Creating binary inputs
-- Validating file
Command failed:
/tmp/arrow-0.11.0.E5KVJ/apache-arrow-0.11.0/js/bin/integration.js -a
/tmp/tmpb5x05ms9/5a1c8b2fdbd642faa581c5279ccc92b5_struct_example.json_to_arrow
-j
/tmp/arrow-0.11.0.E5KVJ/apache-arrow-0.11.0/integration/data/struct_example.json
--mode VALIDATE -t es5 -m umd
With output:
--------------
[14:03:24] Using gulpfile
/tmp/arrow-0.11.0.E5KVJ/apache-arrow-0.11.0/js/gulpfile.js
[14:03:24] Starting 'test'...
[14:03:24] Starting 'test:es5:umd'...
FAIL test/integration/validate-tests.ts
● Integration › Can read multiple tables from the same stream with a
special stream reader
TypeError: Symbol.asyncIterator is not defined.
33 |
34 | const { Table, read } = Arrow;
> 35 | const { fromReadableStream, readBuffersAsync,
readRecordBatchesAsync } = Arrow;
36 |
37 | if (!process.env.JSON_PATHS || !process.env.ARROW_PATHS) {
38 | throw new Error('Integration tests need paths to both
json and arrow files');
at Object.<anonymous>.__asyncGenerator
(test/integration/validate-tests.ts:35:38)
at allTablesReadableStream (test/integration/validate-tests.ts:154:20)
at test/integration/validate-tests.ts:152:42
at test/integration/validate-tests.ts:23:71
at Object.<anonymous>.__awaiter (test/integration/validate-tests.ts:19:12)
at Object.test (test/integration/validate-tests.ts:141:99)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 6 passed, 7 total
Snapshots: 0 total
Time: 0.868s
Ran all test suites matching /test\/integration\/*/i.
[14:03:25] 'test:es5:umd' errored after 1.03 s
[14:03:25] Error: exited with error code: 1
at ChildProcess.onexit
(/tmp/arrow-0.11.0.E5KVJ/apache-arrow-0.11.0/js/node_modules/end-of-stream/index.js:39:36)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
[14:03:25] 'test' errored after 1.03 s
maybe my version of npm is too old? Clearly we need some kind of
nightly integration test that creates a snapshot source release and
tries to verify on common platforms so we get early warnings about
these issues
On Thu, Oct 4, 2018 at 5:23 PM Uwe L. Korn <uw...@xhochy.com> wrote:
>
> I'm struggling a bit to verify the release with the following issues:
>
> * CMake is too old on Ubuntu 16.04, gbenchmark needs a newer one. A
> workaround here would be to install a newer one with conda. Should we do that
> always?
> * Java tests fail on Ubuntu 16.04:
> https://issues.apache.org/jira/browse/ARROW-3443 Following Jacques' comment,
> we should disable/ignore the test for this release.
> * Plasma tests fail for me on OSX with the following error:
> /var/folders/3j/b8ctc4654q71hd_nqqh8yxc00000gp/T/arrow-0.11.0.XXXXX.Psg69q8j/apache-arrow-0.11.0/cpp/src/plasma/io.cc:136:
> Socket pathname is too long.
> /var/folders/3j/b8ctc4654q71hd_nqqh8yxc00000gp/T/arrow-0.11.0.XXXXX.Psg69q8j/apache-arrow-0.11.0/cpp/src/plasma/store.cc:900:
> Check failed: socket >= 0
>
> Uwe
>
> On Thu, Oct 4, 2018, at 12:28 PM, Wes McKinney wrote:
> > Can you add your other GPG key to the KEYS file? If it's a trusted key
> > that seems OK to me
> > On Thu, Oct 4, 2018 at 4:34 AM Kouhei Sutou <k...@clear-code.com> wrote:
> > >
> > > Hi,
> > >
> > > Oh...
> > >
> > > I've resign by correct GPG key and replaced .asc files.
> > >
> > > Can we use this RC? Or should we create one more RC for
> > > fixing this .asc problem?
> > >
> > >
> > > Thanks,
> > > --
> > > kou
> > >
> > > In <1538639225.4190225.1530323248.542da...@webmail.messagingengine.com>
> > > "Re: [VOTE] Release Apache Arrow 0.11.0 (RC1)" on Thu, 04 Oct 2018
> > > 09:47:05 +0200,
> > > "Uwe L. Korn" <uw...@xhochy.com> wrote:
> > >
> > > > Hello Kou,
> > > >
> > > > It seems like you have used a GPG key that is not in the main keys
> > > > files:
> > > >
> > > > ```
> > > > + gpg --verify apache-arrow-0.11.0.tar.gz.asc apache-arrow-0.11.0.tar.gz
> > > > gpg: Signature made Thu Oct 4 05:46:23 2018 CEST
> > > > gpg: using DSA key
> > > > 7714A383F6F73E2D9828791D17423F641C837F31
> > > > gpg: Can't check signature: No public key
> > > > ```
> > > >
> > > > Thus it already aborts for me the archive verification step.
> > > >
> > > > Uwe
> > > >
> > > > On Thu, Oct 4, 2018, at 9:02 AM, Kouhei Sutou wrote:
> > > >> +1 (binding)
> > > >>
> > > >> I ran the followings on Debian GNU/Linux sid:
> > > >>
> > > >> * ARROW_HAVE_GPU=no dev/release/verify-release-candidate.sh source
> > > >> 0.11.0 1
> > > >> * dev/release/verify-release-candidate.sh binaries 0.11.0 1
> > > >>
> > > >> with:
> > > >>
> > > >> * gcc version 8.2.0 (Debian 8.2.0-1)
> > > >> * OpenJDK 1.8.0_181
> > > >> * Ruby 2.6.0dev
> > > >> * NodeJS 10.8.0
> > > >>
> > > >> Note that the verification is failed with ARROW_HAVE_GPU=yes:
> > > >>
> > > >> --
> > > >> pyarrow/tests/
> > > >> test_plasma.py::TestPlasmaClient::test_create_with_metadata ERROR [
> > > >> 81%]
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stderr
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> /tmp/arrow-0.11.0.Y6KZK/apache-arrow-0.11.0/cpp/src/plasma/store.cc:971:
> > > >> Allowing the Plasma store to use up to 0.1GB of memory.
> > > >> /tmp/arrow-0.11.0.Y6KZK/apache-arrow-0.11.0/cpp/src/plasma/
> > > >> store.cc:1001: Starting object store with directory /dev/shm and huge
> > > >> page support disabled
> > > >> munmap_chunk(): invalid pointer
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>
> > > >> self = <pyarrow.tests.test_plasma.TestPlasmaClient object at
> > > >> 0x7fc4201ae710>
> > > >> test_method = <bound method TestPlasmaClient.test_create_with_metadata
> > > >> of <pyarrow.tests.test_plasma.TestPlasmaClient object at
> > > >> 0x7fc4201ae710>>
> > > >>
> > > >> def teardown_method(self, test_method):
> > > >> try:
> > > >> # Check that the Plasma store is still alive.
> > > >> assert self.p.poll() is None
> > > >> # Ensure Valgrind and/or coverage have a clean exit
> > > >> # Valgrind misses SIGTERM if it is delivered before the
> > > >> # event loop is ready; this race condition is mitigated
> > > >> # but not solved by time.sleep().
> > > >> if USE_VALGRIND:
> > > >> time.sleep(1.0)
> > > >> self.p.send_signal(signal.SIGTERM)
> > > >> if sys.version_info >= (3, 3):
> > > >> self.p.wait(timeout=5)
> > > >> else:
> > > >> self.p.wait()
> > > >> > assert self.p.returncode == 0
> > > >> E assert -6 == 0
> > > >> E + where -6 = <subprocess.Popen object at
> > > >> 0x7fc4201aef98>.returncode
> > > >> E + where <subprocess.Popen object at 0x7fc4201aef98> =
> > > >> <pyarrow.tests.test_plasma.TestPlasmaClient object at 0x7fc4201ae710>.p
> > > >>
> > > >> pyarrow/tests/test_plasma.py:141: AssertionError
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> > /tmp/arrow-0.11.0.Y6KZK/apache-arrow-0.11.0/python/pyarrow/tests/test_plasma.py(141)teardown_method()
> > > >> -> assert self.p.returncode == 0
> > > >> (Pdb)
> > > >> --
> > > >>
> > > >> My GPU is GeForce GTK 1050Ti.
> > > >>
> > > >> I don't know much about Plasma and GPU stability but I don't
> > > >> think that this is a blocker.
> > > >>
> > > >>
> > > >> Thanks,
> > > >> --
> > > >> kou
> > > >>
> > > >> In <20181004.142257.2091714912323248008....@clear-code.com>
> > > >> "[VOTE] Release Apache Arrow 0.11.0 (RC1)" on Thu, 04 Oct 2018
> > > >> 14:22:57 +0900 (JST),
> > > >> Kouhei Sutou <k...@clear-code.com> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > I'd like to propose the 1st release candidate (rc1) of
> > > >> > Apache Arrow version 0.11.0 (rc0 is never voted on due to
> > > >> > problems I discovered while releasing). This is a major
> > > >> > release consisting of 283 resolved JIRAs [1].
> > > >> >
> > > >> > The source release rc1 is hosted at [2].
> > > >> >
> > > >> > This release candidate is based on commit
> > > >> > 18a61f6758767169a35745f8646991ed245d5a82 [3]
> > > >> >
> > > >> > The changelog is located at [4].
> > > >> >
> > > >> > Please download, verify checksums and signatures, run the unit tests,
> > > >> > and vote on the release.
> > > >> >
> > > >> > See [5] for how to validate a release candidate.
> > > >> >
> > > >> > The vote will be open for at least 72 hours.
> > > >> >
> > > >> > [ ] +1 Release this as Apache Arrow 0.11.0
> > > >> > [ ] +0
> > > >> > [ ] -1 Do not release this as Apache Arrow 0.11.0 because...
> > > >> >
> > > >> >
> > > >> > Thanks,
> > > >> > --
> > > >> > kou
> > > >> >
> > > >> > [1]:
> > > >> > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.11.0
> > > >> > [2]:
> > > >> > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-0.11.0-rc1/
> > > >> > [3]:
> > > >> > https://github.com/apache/arrow/commit/18a61f6758767169a35745f8646991ed245d5a82
> > > >> > [4]:
> > > >> > https://github.com/apache/arrow/blob/7940ffe559810fec82cb2fbb0b13f5809cb5fe85/CHANGELOG.md
> > > >> > [5]:
> > > >> > https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates
