[DISCUSS] Use conbench.ursa.dev for arrow-rs and arrow-datafusion

2021-09-10 Thread QP Hou
Hi, I think conbench.ursa.dev works really well for the main arrow repo, especially the ability to request on demand benchmarks during PR reviews by mentioning usrabot in a comment. I am wondering if it is something that arrow-rs and arrow-datafusion could leverage as well to help speed up PR rev

Re: [ANNOUNCE] New Arrow committer: Nic Crane

2021-09-10 Thread Nic Crane
Thanks everyone! :) On Fri, Sep 10, 2021 at 4:54 AM QP Hou wrote: > Congrats Nic! > > On Thu, Sep 9, 2021 at 8:47 AM Neal Richardson > wrote: > > > > On behalf of the Apache Arrow PMC, I'm happy to announce that Nic Crane > > has accepted an invitation to become a committer on Apache Arrow. > >

[Rust] [DataFusion] Move Statistics and Cost Based Optimizations to physical plans

2021-09-10 Thread Andrew Lamb
I would like to draw some attention to a PR [1] that proposes to move statistics (and the cost based optimizations that rely on them) into the physical planning realm. The feedback so far on this change seems positive but since it may have non-trivial architectural impact on downstream projects th

[VOTE][RUST] Release Apache Arrow Rust 5.4.0 RC1

2021-09-10 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 5.4.0. NOTE: There is one known issue [5] that causes one of the CI tests to fail but this failure appears to be an artifact of the CI setup rather than something wrong with the code itself. This release candidate

Re: Arrow in HPC

2021-09-10 Thread Micah Kornfield
> > I would support doing the work necessary to get UCX (or really any other > transport) supported, even if it is a lot of work. (I'm hoping this clears > the path to supporting a Flight-to-browser transport as well; a few > projects seem to have rolled their own approaches but I think Flight itse

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Keith Kraus
Apologies, libstdc++ is what I meant (too used to arguing about glibc compatibility). Devtoolset forcibly disables the `_GLIBCXX_USE_CXX11_ABI` flag because the mixed linkage model it uses can't support it: https://bugzilla.redhat.com/show_bug.cgi?id=1546704. RHEL / CentOS 8 does support the new AB

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Antoine Pitrou
Le 10/09/2021 à 17:05, Keith Kraus a écrit : For what it's worth, setting it to 1 as opposed to 0 will make the package incompatible with CentOS / RHEL 7 as the glibc they ship does not support the new ABI. It is not about the glibc, it's about the stdlibc++. -Keith On Fri, Sep 10, 2021,

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Keith Kraus
For what it's worth, setting it to 1 as opposed to 0 will make the package incompatible with CentOS / RHEL 7 as the glibc they ship does not support the new ABI. -Keith On Fri, Sep 10, 2021, 4:53 AM Philipp Moritz wrote: > Ah ok, that makes sense! I'm also not even sure if > _GLIBCXX_USE_CXX11_

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
Ah ok, that makes sense! I'm also not even sure if _GLIBCXX_USE_CXX11_ABI=0 was ever mandated on manylinux1, it might just be a community convention. I posted https://discuss.python.org/t/how-to-set-glibcxx-use-cxx11-abi-for-manylinux2014-and-manylinux2010-wheels/10551 , we can shift the discussio

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Antoine Pitrou
Le 10/09/2021 à 10:05, Philipp Moritz a écrit : Thanks for your answer Antoine! Considering your first comment, there is a section in https://www.python.org/dev/peps/pep-0571 under "Backwards compatibility with manylinux1 wheels" that states "manylinux1 wheels are considered manylinux2010 whee

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
Thanks for your answer Antoine! Considering your first comment, there is a section in https://www.python.org/dev/peps/pep-0571 under "Backwards compatibility with manylinux1 wheels" that states "manylinux1 wheels are considered manylinux2010 wheels" and the same remark in https://www.python.org/de

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Antoine Pitrou
Le 10/09/2021 à 09:12, Philipp Moritz a écrit : Dear all, how do you think _GLIBCXX_USE_CXX11_ABI should be set for manylinux2014 (and manylinux2010) wheels? Should it be 0 or 1? Unfortunately I don't see https://www.python.org/dev/peps/pep-0599/ or https://github.com/pypa/manylinux specifying

[Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
Dear all, how do you think _GLIBCXX_USE_CXX11_ABI should be set for manylinux2014 (and manylinux2010) wheels? Should it be 0 or 1? Unfortunately I don't see https://www.python.org/dev/peps/pep-0599/ or https://github.com/pypa/manylinux specifying it. I think for manylinux1 the common wisdom was to