Re: [Java] C Data Interface and dictionaries

2021-08-26 Thread roee shlomo
> It seems that we have both raw value and encoded value types in the Java implementation, so there is no information loss? I think that in the Java memory format they are both the index type, see https://github.com/apache/arrow/blob/5003278ded77f1ab385425143aafd085fda1f701/java/vector/src/main/ja

Re: [DISCUSS] Developing an "Arrow Compute IR [Intermediate Representation]" to decouple language front ends from Arrow-native compute engines

2021-08-26 Thread Phillip Cloud
Thanks for the feedback Jacques, very helpful. In the latest version of the PR, I've tried to incorporate nearly all of these points. - I've incorporated most of what you had for dereferencing operations into the PR, and gotten rid of schemas except on Read/Write relations. - With respect to prope

[VOTE][RUST] Release Apache Arrow Rust 5.3.0 RC1

2021-08-26 Thread Andrew Lamb
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 5.3.0. This release candidate is based on commit: 9f7707cd2c0fe07e989dbd0afce4d4850921df4a [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, ver

Re: [DISCUSS] Developing an "Arrow Compute IR [Intermediate Representation]" to decouple language front ends from Arrow-native compute engines

2021-08-26 Thread Micah Kornfield
As an FYI, Iceberg is also considering an IR in relation to view support [1]. I chimed in and pointed them to this thread and Wes's doc. Phillip and Jacques chimed in there as well. [1] https://mail-archives.apache.org/mod_mbox/iceberg-dev/202108.mbox/%3CCAKRVfm6h6WxQtp5fj8Yj8XWR1wFe8VohOkPuoZZG