[RESULT][VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-16 Thread Raúl Cumplido
Hi, Thanks everyone. The result of the vote was successful with 3 +1 binding votes, 4 +1 non-binding votes and no -1 votes. +1 (binding): David Li, Raúl Cumplido, Sutou Kouhei +1 (non binding): Ruoxi Sun, Jean-Baptiste Onofré, Gang Wu, Fokko Driesprong I will start the post release tasks for 17.

[DISCUSS] 8-bit Boolean Canonical Extension Type

2024-07-16 Thread Joel Lubinitsky
Hi Arrow devs, I'm working on adding an extension type for 8-bit booleans, and wanted to start a discussion about it here because it could be valuable to others if adopted as a canonical extension type. The native implementation of the Boolean type uses 1 bit to encode each value, enabling a very

Re: [DISCUSS] 8-bit Boolean Canonical Extension Type

2024-07-16 Thread Ian Cook
Thanks for taking the initiative on this! As demonstrated by [1], the wish for an 8-bit Boolean extension type is long-standing. I think this is a worthwhile addition to Arrow's canonical extension types. Before the vote, I would like to see verification that this truly enables zero-copy to/from

Re: [DISCUSS][C++] Empty directory marker creation in S3FileSystem

2024-07-16 Thread Antoine Pitrou
Hello Aldrin, It's not either/or, the directory marker is created everytime necessary, for example when CreateDir() is called. Regards Antoine. Le 15/07/2024 à 19:20, Aldrin a écrit : Thanks Antoine! Preserving the property across multiple clients (and presumably across independent ses

Re: [DISCUSS] 8-bit Boolean Canonical Extension Type

2024-07-16 Thread Antoine Pitrou
Hi Joel, This looks good to me on the principle. Can you split the spec and the implementation(s) into separate PRs? Regards Antoine. Le 16/07/2024 à 13:18, Joel Lubinitsky a écrit : Hi Arrow devs, I'm working on adding an extension type for 8-bit booleans, and wanted to start a discuss

Re: Understanding possible synergies between arrow & zarr communities?

2024-07-16 Thread Antoine Pitrou
Hi Carl, Le 08/07/2024 à 18:43, Carl Boettiger a écrit : As an observer to both communities, I'm interested in if there is or might be more communication between the Pangeo community's focus on Zarr serialization with what the Arrow team has done with Parquet. I recognize that these are diff

Re: [RESULT][VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-16 Thread Driesprong, Fokko
Hey Raúl, First of all, thanks for taking care of the release. At PyIceberg we were waiting for some of the fixes, so we upgraded right away. However, it seems that the Python 3.9 artifacts are missing from PyPi: - https://pypi.org/project/pyarrow/17.0.0/#files - https://pypi.org/project/py

Re: [RESULT][VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-16 Thread Raúl Cumplido
Hi Fokko, Yes, I shared this on Zulip but responding here to get more visibility, there's been an issue uploading wheels to PyPI due to project size exceeding our current quota there. I've requested a quota increase to 40GiB here [1]. I'll upload the missing wheels once this is done. Sorry for t

Re: [RESULT][VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-16 Thread Raúl Cumplido
As a clarification, some wheels for Python 3.8 are also missing. It failed after uploading all the wheels for 3.{10,11,12} and in the middle of 3.8. El mar, 16 jul 2024 a las 18:52, Raúl Cumplido () escribió: > > Hi Fokko, > > Yes, I shared this on Zulip but responding here to get more > visibilit

Re: [RESULT][VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-16 Thread Fokko Driesprong
Thanks Raúl for the quick response. Next to some of the wheels, the source distribution is also missing. Thanks for working on this. Kind regards, Fokko Op di 16 jul 2024 om 18:55 schreef Raúl Cumplido : > As a clarification, some wheels for Python 3.8 are also missing. It > failed after uploadi

Re: [DISCUSS][C++] Empty directory marker creation in S3FileSystem

2024-07-16 Thread Aldrin
Ah, okay. Then, I suppose that an approach between 1 and 2 makes some sense to me: add an option to disable creating the marker on object deletion/removal. I think this alone isn't the best solution but it seems to at least add a mode where creating the marker is more controlled. As an aside, ar

Re: [Discuss][C++] Switch to mimalloc by default?

2024-07-16 Thread Antoine Pitrou
Hello, Thanks all for this discussion. Given that there was no strong argument against doing this, I decided to move forward and the change was made in https://github.com/apache/arrow/pull/40875 Regards Antoine. On Wed, 5 Jun 2024 17:18:36 +0200 Antoine Pitrou wrote: > Hello, > > Arrow C++