Well this is not good and pretty disappointing given that we had nearly a month to sort through the implications of Micah’s patch. We should try to resolve this ASAP
On Sun, Jul 19, 2020 at 5:10 PM Bryan Cutler <cutl...@gmail.com> wrote: > +0 (non-binding) > > I ran verification script for binaries and then source, as below, and both > look good > ARROW_TMPDIR=/tmp/arrow-test TEST_DEFAULT=0 TEST_SOURCE=1 TEST_CPP=1 > TEST_PYTHON=1 TEST_JAVA=1 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 > dev/release/verify-release-candidate.sh source 1.0.0 1 > > I tried to patch Spark locally to verify the recent change in nested > timestamps and was not able to get things working quite right, but I'm not > sure if the problem is in Spark, Arrow or my patch - hence my vote of +0. > > Here is what I'm seeing > > ``` > (Input as datetime) > datetime.datetime(2018, 3, 10, 0, 0) > datetime.datetime(2018, 3, 15, 0, 0) > > (Struct Array) > -- is_valid: all not null > -- child 0 type: timestamp[us, tz=America/Los_Angeles] > [ > 2018-03-10 00:00:00.000000, > 2018-03-10 00:00:00.000000 > ] > -- child 1 type: timestamp[us, tz=America/Los_Angeles] > [ > 2018-03-15 00:00:00.000000, > 2018-03-15 00:00:00.000000 > ] > > (Flattened Arrays) > types [TimestampType(timestamp[us, tz=America/Los_Angeles]), > TimestampType(timestamp[us, tz=America/Los_Angeles])] > [<pyarrow.lib.TimestampArray object at 0x7ffbbd88f520> > [ > 2018-03-10 00:00:00.000000, > 2018-03-10 00:00:00.000000 > ], <pyarrow.lib.TimestampArray object at 0x7ffba958be50> > [ > 2018-03-15 00:00:00.000000, > 2018-03-15 00:00:00.000000 > ]] > > (Pandas Conversion) > [ > 0 2018-03-09 16:00:00-08:00 > 1 2018-03-09 16:00:00-08:00 > dtype: datetime64[ns, America/Los_Angeles], > > 0 2018-03-14 17:00:00-07:00 > 1 2018-03-14 17:00:00-07:00 > dtype: datetime64[ns, America/Los_Angeles]] > ``` > > Based on output of existing a correct timestamp udf, it looks like the > pyarrow Struct Array values are wrong and that's carried through the > flattened arrays, causing the Pandas values to have a negative offset. > > Here is output from a working udf with timestamp, the pyarrow Array > displays in UTC time, I believe. > > ``` > (Timestamp Array) > type timestamp[us, tz=America/Los_Angeles] > [ > [ > 1969-01-01 09:01:01.000000 > ] > ] > > (Pandas Conversion) > 0 1969-01-01 01:01:01-08:00 > Name: _0, dtype: datetime64[ns, America/Los_Angeles] > > (Timezone Localized) > 0 1969-01-01 01:01:01 > Name: _0, dtype: datetime64[ns] > ``` > > I'll have to dig in further at another time and debug where the values go > wrong. > > On Sat, Jul 18, 2020 at 9:51 PM Micah Kornfield <emkornfi...@gmail.com> > wrote: > > > +1 (binding) > > > > Ran wheel and binary tests on ubuntu 19.04 > > > > On Fri, Jul 17, 2020 at 2:25 PM Neal Richardson < > > neal.p.richard...@gmail.com> > > wrote: > > > > > +1 (binding) > > > > > > In addition to the usual verification on > > > https://github.com/apache/arrow/pull/7787, I've successfully staged > the > > R > > > binary artifacts on Windows ( > > > https://github.com/r-windows/rtools-packages/pull/126), macOS ( > > > https://github.com/autobrew/homebrew-core/pull/12), and Linux ( > > > https://github.com/ursa-labs/arrow-r-nightly/actions/runs/172977277) > > using > > > the release candidate. > > > > > > And I agree with the judgment about skipping a JS release artifact. > Looks > > > like there hasn't been a code change since October so there's no point. > > > > > > Neal > > > > > > On Fri, Jul 17, 2020 at 10:37 AM Wes McKinney <wesmck...@gmail.com> > > wrote: > > > > > > > I see the JS failures as well. I think it is a failure localized to > > > > newer Node versions since our JavaScript CI works fine. I don't think > > > > it should block the release given the lack of development activity in > > > > JavaScript [1] -- if any JS devs are concerned about publishing an > > > > artifact then we can skip pushing it to NPM > > > > > > > > @Ryan it seems it may be something environment related on your > > > > machine, I'm on Ubuntu 18.04 and have not seen this. > > > > > > > > On > > > > > > > > > * Python 3.8 wheel's tests are failed. 3.5, 3.6 and 3.7 > > > > > are passed. It seems that -larrow and -larrow_python for > > > > > Cython are failed. > > > > > > > > I suspect this is related to > > > > > > > > > > > > > > https://github.com/apache/arrow/commit/120c21f4bf66d2901b3a353a1f67bac3c3355924#diff-0f69784b44040448d17d0e4e8a641fe8 > > > > , > > > > but I don't think it's a blocking issue > > > > > > > > [1]: https://github.com/apache/arrow/commits/master/js > > > > > > > > On Fri, Jul 17, 2020 at 9:42 AM Ryan Murray <rym...@dremio.com> > wrote: > > > > > > > > > > I've tested Java and it looks good. However the verify script keeps > > on > > > > > bailing with protobuf related errors: > > > > > 'cpp/build/orc_ep-prefix/src/orc_ep-build/c++/src/orc_proto.pb.cc' > > and > > > > > friends cant find protobuf definitions. A bit odd as cmake can see > > > > protobuf > > > > > headers and builds directly off master work just fine. Has anyone > > else > > > > > experienced this? I am on ubutnu 18.04 > > > > > > > > > > On Fri, Jul 17, 2020 at 10:49 AM Antoine Pitrou < > anto...@python.org> > > > > wrote: > > > > > > > > > > > > > > > > > +1 (binding). I tested on Ubuntu 18.04. > > > > > > > > > > > > * Wheels verification went fine. > > > > > > * Source verification went fine with CUDA enabled and > > > > > > TEST_INTEGRATION_JS=0 TEST_JS=0. > > > > > > > > > > > > I didn't test the binaries. > > > > > > > > > > > > Regards > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > Le 17/07/2020 à 03:41, Krisztián Szűcs a écrit : > > > > > > > Hi, > > > > > > > > > > > > > > I would like to propose the second release candidate (RC1) of > > > Apache > > > > > > > Arrow version 1.0.0. > > > > > > > This is a major release consisting of 826 resolved JIRA > > issues[1]. > > > > > > > > > > > > > > The verification of the first release candidate (RC0) has > failed > > > > [0], and > > > > > > > the packaging scripts were unable to produce two wheels. > Compared > > > > > > > to RC0 this release candidate includes additional patches for > the > > > > > > > following bugs: ARROW-9506, ARROW-9504, ARROW-9497, > > > > > > > ARROW-9500, ARROW-9499. > > > > > > > > > > > > > > This release candidate is based on commit: > > > > > > > bc0649541859095ee77d03a7b891ea8d6e2fd641 [2] > > > > > > > > > > > > > > The source release rc1 is hosted at [3]. > > > > > > > The binary artifacts are hosted at [4][5][6][7]. > > > > > > > The changelog is located at [8]. > > > > > > > > > > > > > > Please download, verify checksums and signatures, run the unit > > > tests, > > > > > > > and vote on the release. See [9] for how to validate a release > > > > candidate. > > > > > > > > > > > > > > The vote will be open for at least 72 hours. > > > > > > > > > > > > > > [ ] +1 Release this as Apache Arrow 1.0.0 > > > > > > > [ ] +0 > > > > > > > [ ] -1 Do not release this as Apache Arrow 1.0.0 because... > > > > > > > > > > > > > > [0]: > > > > https://github.com/apache/arrow/pull/7778#issuecomment-659065370 > > > > > > > [1]: > > > > > > > > > > > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%201.0.0 > > > > > > > [2]: > > > > > > > > > > > > > > > > https://github.com/apache/arrow/tree/bc0649541859095ee77d03a7b891ea8d6e2fd641 > > > > > > > [3]: > > > > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-1.0.0-rc1 > > > > > > > [4]: https://bintray.com/apache/arrow/centos-rc/1.0.0-rc1 > > > > > > > [5]: https://bintray.com/apache/arrow/debian-rc/1.0.0-rc1 > > > > > > > [6]: https://bintray.com/apache/arrow/python-rc/1.0.0-rc1 > > > > > > > [7]: https://bintray.com/apache/arrow/ubuntu-rc/1.0.0-rc1 > > > > > > > [8]: > > > > > > > > > > > > > > > > https://github.com/apache/arrow/blob/bc0649541859095ee77d03a7b891ea8d6e2fd641/CHANGELOG.md > > > > > > > [9]: > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates > > > > > > > > > > > > > > > > > > > > > > >
