[Rust] Recent backward incompatible changes in master

2021-01-19 Thread Jorge Cardoso Leitão
Hi, Just a heads up that there are 4 changes in master that may affect your code, and I would like to bring them to your attention. 1. `memory::allocate_aligned` no longer initializes memory regions with zeros. Use `memory::allocate_aligned_zeroed` for this. Rational: up to 3.0, we allocate a ze

Re: [Proposal] Modify release process to vote only on source release

2021-01-19 Thread Neal Richardson
Agreed, there are multiple issues to resolve in order for our release process to be manageable and scalable for the project. This procedural change is not a silver bullet, and if we agree to it, it doesn't mean that our releases are "fixed". But it's the only change where the solution is a discussi

Re: [Proposal] Modify release process to vote only on source release

2021-01-19 Thread Wes McKinney
I'm OK with moving to source only releases, but we need to take a step back and consider how our CI/CD is failing to notify us in a suitably timely and automated way about the packages being broken. For example, the fact that we had 2 failed RCs as the result of packaging issues points to a broken

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Sutou Kouhei
Hi, We can't use DYLD_LIBRARY_PATH in our verification script because it's not passed to subprocesses by security reason. GLib doesn't support using @rpath-ed libraries for now. Whether using @rpath or not is determined when the library is compiled. So it's important how did you install your Snap

Re: [C++] Shall we modify the ORC reader?

2021-01-19 Thread Deepak Majeti
Hi Ying, I can help review/merge any ORC C++ contributions. On Thu, Jan 14, 2021 at 6:57 PM Ying Zhou wrote: > Well, I haven’t found any. Thankfully ORC does work and I can figure out > how it works by testing using simple examples. However I have never managed > to contact the ORC community a

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Ying Zhou
Hi, My local version of Snappy is in /usr/local/lib as libsnappy.1.dylib . My DYLD_LIBRARY_PATH does have /usr/local/lib. > On Jan 19, 2021, at 7:57 PM, Sutou Kouhei wrote: > > Hi, > >> dyld: Library not loaded: @rpath/libsnappy.1.dylib >> Referenced from: >> /private/var/folders/yb/dc13kd1

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Sutou Kouhei
Hi, > dev/release/verify-release-candidate.sh: line 95: gpg: command not found You need gpg command to verify sign. Could you install gpg command and try again? Thanks, -- kou In "Re: [VOTE] Release Apache Arrow 3.0.0 - RC2" on Tue, 19 Jan 2021 20:09:41 -0500, Andrew Wieteska wrote: >

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Andrew Wieteska
Hi all, I'm not able to verify on MacOS 10.15.7: (base) andrewwieteska@Andrews-MacBook-Pro arrow % dev/release/verify-release-candidate.sh source 3.0.0 2 + set -o pipefail +++ dirname dev/release/verify-release-candidate.sh ++ cd dev/release ++ pwd + SOURCE_DIR=/Users/andrewwieteska/repos/ar

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Sutou Kouhei
Hi, > dyld: Library not loaded: @rpath/libsnappy.1.dylib > Referenced from: > /private/var/folders/yb/dc13kd1552vc_x61qzpgmtjhgn/T/arrow-3.0.0.X.00Y3BiKD/install/lib/libarrow.300.0.0.dylib > Reason: image not found How did you install your Snappy? Thanks, -- kou In "Re: [VOTE]

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Ying Zhou
There are definitely dependencies issues in at least GLib. I’m going to turn off Glib and see whether other issues exist. + make -j4 /Library/Developer/CommandLineTools/usr/bin/make all-recursive Making all in arrow-glib GEN stamp-enums.h GEN stamp-enums.c touch stamp-enums.c touch

Arrow sync call January 20 at 12:00 US/Eastern, 17:00 UTC

2021-01-19 Thread Neal Richardson
Hi all, Reminder that our biweekly call is tomorrow at https://meet.google.com/vtm-teks-phx. All are welcome to join. Notes will be shared with the mailing list afterward. Neal

[Proposal] Modify release process to vote only on source release

2021-01-19 Thread Neal Richardson
Hi all, Over the past year, there's been a lot of discussion around the challenges we face as a project in doing releases. Because they are costly to do, we don't do them often; because we don't do them often, they become even costlier. There are only a small number of people (PMC members with GPG

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Andy Grove
+1 (binding) I verified the checksum and ran the Rust tests on Ubuntu 18.04. On Tue, Jan 19, 2021 at 6:49 AM Andrew Lamb wrote: > +1 (non binding) > > I downloaded the source, verified the checksums, and ran the Rust tests on > Mac (OSx 10.15.7) and I think the release is good to go from my > p

Re: [Java] PR review for ARROW-11173

2021-01-19 Thread Bryan Cutler
Hi Nick, I left a note in the PR that I will try to review soon, thanks! On Sun, Jan 17, 2021 at 8:22 PM Nick Bruno wrote: > Hi All, > > I'd like to get feedback on the pull request I created a little over a > week ago - https://github.com/apache/arrow/pull/9151 > > It adds support for Map type

Re: [VOTE] Release Apache Arrow 3.0.0 - RC0

2021-01-19 Thread Ying Zhou
Oh I see. Yup. Thanks, Ying > On Jan 19, 2021, at 7:56 AM, Antoine Pitrou wrote: > > > Plasma is deprecated and unmaintained, I don't think we should hold the > release for that. > > Regards > > Antoine. > > > Le 19/01/2021 à 13:21, Ying Zhou a écrit : >> Yup this can be modified. Now for

Re: Is it ok to merge PRs to apache/master now given the 3.0.0 release?

2021-01-19 Thread Andrew Lamb
Ok -- thank you (on both accounts) Andrew On Tue, Jan 19, 2021 at 9:42 AM Wes McKinney wrote: > I don't see any need to stop merging PRs. I recommend you go ahead and > if Krisztian needs to cut another RC he can do it out of a branch. > I've said this on some other e-mail threads and there hav

Re: Is it ok to merge PRs to apache/master now given the 3.0.0 release?

2021-01-19 Thread Wes McKinney
I don't see any need to stop merging PRs. I recommend you go ahead and if Krisztian needs to cut another RC he can do it out of a branch. I've said this on some other e-mail threads and there haven't been objections so I think it's okay. We can formalize this policy after 3.0.0 so there isn't this

Is it ok to merge PRs to apache/master now given the 3.0.0 release?

2021-01-19 Thread Andrew Lamb
There are 40 PRs outstanding for Rust at the moment (though not all are ready to merge): https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+label%3Alang-rust Should we continue to wait to merge these PRs until the 3.0.0 release is approved? I have been assuming that holding off merging the

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-19 Thread Andrew Lamb
+1 (non binding) I downloaded the source, verified the checksums, and ran the Rust tests on Mac (OSx 10.15.7) and I think the release is good to go from my perspective. Thanks for all the work to get this far. On Mon, Jan 18, 2021 at 10:49 PM Krisztián Szűcs wrote: > Hi, > > I would like to pro

Re: [VOTE] Release Apache Arrow 3.0.0 - RC0

2021-01-19 Thread Antoine Pitrou
Plasma is deprecated and unmaintained, I don't think we should hold the release for that. Regards Antoine. Le 19/01/2021 à 13:21, Ying Zhou a écrit : > Yup this can be modified. Now for RC2 we have a new error on macOS Catalina. > > pyarrow/tests/test_plasma.py::TestPlasmaClient::test_use_fu

Re: [VOTE] Release Apache Arrow 3.0.0 - RC0

2021-01-19 Thread Ying Zhou
Yup this can be modified. Now for RC2 we have a new error on macOS Catalina. pyarrow/tests/test_plasma.py::TestPlasmaClient::test_use_full_memory Fatal Python error: Aborted Here is what errored out: for _ in range(100): create_object( self.plasma_client2,

[NIGHTLY] Arrow Build Report for Job nightly-2021-01-19-0

2021-01-19 Thread Crossbow
Arrow Build Report for Job nightly-2021-01-19-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-19-0 Failed Tasks: - centos-8-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-19-0-travis-centos-8-aarch64 - con