Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow

2022-10-17 Thread Andrew Lamb
And the final installment: https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/ On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb wrote: > Second post is now live (this time with non localhost url :thumbsup:): > > https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-

[RESULT][VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-17 Thread Andrew Lamb
With 7 +1 votes (5 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-25.0.0 I have also published the release on crates.io Thank you all for another successful release Andrew On Sun, Oct 16, 2022 at 4:43 PM Neville Dip

Re: [VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-17 Thread Andrew Lamb
Hi Andy, I could not reproduce this. I ran the verification script locally, with the following commands (Mac x86_64) and it passed for me: tar xf apache-arrow-rs-25.0.0.tar.gz cd apache-arrow-rs-25.0.0 ./dev/release/verify-release-candidate.sh 25.0.0 1 I wonder if the clone of one of the test da

Re: [DISCUSS] Integrate existing Spark connector for Flight

2022-10-17 Thread Antoine Pitrou
Le 17/10/2022 à 21:27, David Li a écrit : Hey Matt, This is cool to see. To be clear, this is an implementation of Spark DataSourceV2 using Arrow Flight? I think the questions I have are: - Does this belong under Arrow, or under Spark - I lean towards it being closer to Spark than Arrow;

Re: [DISCUSS] Integrate existing Spark connector for Flight

2022-10-17 Thread David Li
Hey Matt, This is cool to see. To be clear, this is an implementation of Spark DataSourceV2 using Arrow Flight? I think the questions I have are: - Does this belong under Arrow, or under Spark - I lean towards it being closer to Spark than Arrow; - How would we do the testing with Spark; - Wha

[DISCUSS] Integrate existing Spark connector for Flight

2022-10-17 Thread Matt Phelps
Hi Arrow team, I’d like to make a contribution to the Arrow project by submitting an existing Spark connector for Flight. Ryan Murray and Kyle Brooks wrote a Flight implementation (“flight-spark”) of Spark’s DataSource V2 interface found here: https://github.com/rymurr/flight-spark-source. We w

Re: [codegen] Dealing with conflicting names in target language

2022-10-17 Thread -
Thank you Antoine, I suspected as much :) Will have to dig into that. For future reference I'm basing my work off this : https://github.com/dcastro/haskell-flatbuffers On Mon, 17 Oct 2022 at 15:25, Antoine Pitrou wrote: > > Hi Marco, > > Le 16/10/2022 à 21:12, - a écrit : > > Hi all, > > > > I

[Rust] Planning for the next Ballista release (0.9.0)

2022-10-17 Thread Andy Grove
I am planning on creating Ballista 0.9.0 RC1 this Friday, October 21st. Here is the tracking issue with a list of issues that I am hoping can be resolved by then. https://github.com/apache/arrow-ballista/issues/273 Let me know (either here on the mailing list or in the issue) if there are additio

Re: [codegen] Dealing with conflicting names in target language

2022-10-17 Thread Antoine Pitrou
Hi Marco, Le 16/10/2022 à 21:12, - a écrit : Hi all, I've noticed the Arrow schema [1] defines some table types that clash with primitive type names in the language I'm targeting. For instance, we see `table Int{}` and `table Bool{}` , both of which are primitive types in Haskell. Are there

[codegen] Dealing with conflicting names in target language

2022-10-17 Thread -
Hi all, I've noticed the Arrow schema [1] defines some table types that clash with primitive type names in the language I'm targeting. For instance, we see `table Int{}` and `table Bool{}` , both of which are primitive types in Haskell. Are there mechanisms in the Arrow standard that let one byp