Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Alenka Frim
> I would say "the data type of individual tensor elements". > (so that people don't try to make it e.g. List(float64)). Also, I don't think any reference to pyarrow should be made here. Good catch! I have updated the text with: * **value_type** is the data type of individual tensor elements

Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1

2023-02-21 Thread Joris Van den Bossche
On Wed, 22 Feb 2023 at 00:55, Sutou Kouhei wrote: > > Hi, > > In > "Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1" on Thu, 16 Feb 2023 > 09:19:50 +0100, > Joris Van den Bossche wrote: > > > current directory: > > /tmp/adbc-verification/apache-arrow-adbc-0.2.0/glib/vendor/bundle/ruby/3.1.

Re: [DISCUSS] arrow/arrow2 path forward

2023-02-21 Thread Micah Kornfield
Great to see, thank you for bringing it to the attention of the wider community. On Mon, Feb 20, 2023 at 4:15 AM Andrew Lamb wrote: > There has been a significant amount of discussion in the past on this list > about the relationship between the two major Rust implementations of > Apache Arrow (

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Dewey Dunnington
+1! I put together a quick R implementation as well to see how the permutation field fits with our native column-major storage [1]. It worked great! Thank you for all of your work assembling all of our collective opinions on this :-) [1] https://gist.github.com/paleolimbot/c42f068c2b8b98255dbfbe37

Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1

2023-02-21 Thread Sutou Kouhei
Hi, In "Re: [VOTE] Release Apache Arrow ADBC 0.2.0 - RC1" on Thu, 16 Feb 2023 09:19:50 +0100, Joris Van den Bossche wrote: > current directory: > /tmp/adbc-verification/apache-arrow-adbc-0.2.0/glib/vendor/bundle/ruby/3.1.0/gems/fiddle-1.1.1/ext/fiddle > make DESTDIR\= install > make: /usr/

Re: Arrow community meeting February 15 at 17:00 UTC

2023-02-21 Thread Sutou Kouhei
Hi, > nanoarrow release > - Dewey might need some help with signing keys I can sign (and upload to https://dist.apache.org/repos/dist/ ). Please let me know when it's needed. Thanks, -- kou In "Re: Arrow community meeting February 15 at 17:00 UTC" on Mon, 20 Feb 2023 11:18:13 -0700, Ian

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Joris Van den Bossche
On Tue, 21 Feb 2023 at 18:00, Rok Mihevc wrote: > > > > > Should we rule that `dim_names` and `permutation` are mutually exclusive? > > > > Since `dim_names` have to "map to the physical layout (row-major)" that > means permutation will always be trivial which indeed makes it unnecessary > to stor

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Rok Mihevc
> > Should we rule that `dim_names` and `permutation` are mutually exclusive? > Since `dim_names` have to "map to the physical layout (row-major)" that means permutation will always be trivial which indeed makes it unnecessary to store both. (This makes me think about extension type implementation

Re: [VOTE][RUST][DataFusion] Release DataFusion Python Bindings 0.8.0 RC1

2023-02-21 Thread Andy Grove
The issue is that the verification script is running "cargo build" instead of "maturin develop". I have created a PR to fix this [1] and will cut a new RC once this is merged. Thanks everyone for helping with this. [1] https://github.com/apache/arrow-datafusion-python/pull/200 On Mon, Feb 20,

Re: [VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Antoine Pitrou
Hi Alenka, Le 21/02/2023 à 13:38, Alenka Frim a écrit : Fixed shape tensor == * Extension name: `arrow.fixed_shape_tensor`. * The storage type of the extension: ``FixedSizeList`` where: * **value_type** is the data type of individual tensors and is an instance of ``

[VOTE] Format: Fixed shape tensor Canonical Extension Type

2023-02-21 Thread Alenka Frim
Hi all, I would like to propose we vote on adding the fixed shape tensor canonical extension type with the following specification: Fixed shape tensor == * Extension name: `arrow.fixed_shape_tensor`. * The storage type of the extension: ``FixedSizeList`` where: * **value_type