Re: [DISCUSS] New Spark Connect Client repository for Rust language

Hi Niranjan, Thanks for jumping in here. I have some experience with monorepos in OSS, more precisely Avro, and I believe this complicates things. Combining everything complicates the build process, and I often find myself fixing things in the PHP client while we are trying to get a Java patch rel

Re: [VOTE] SPIP: Add the TIME data type

+1 (non-binding) Thanks Max! Kind regards, Fokko Op zo 23 feb 2025 om 19:35 schreef Ángel : > + 1 (non-binding) > > El dom, 23 feb 2025, 16:51, Max Gekk escribió: > >> Hi Spark devs, >> >> Following the discussion [1], I'd like to start the vote for the SPIP >> [2]. The SPIP aims to add a new

Re: [DISCUSS] Move Variant to Parquet?

Hey everyone, I agree the Parquet project is a good place to host and evolve the spec (we could store it in parquet-variant?). We would need to align this with the Parquet project. Anyway, I'm familiar both with Iceberg and Parquet and happy to help where needed. Kind regards, Fokko Op ma 19 au

Re: Data Contracts

Hey Phillip, Thanks for raising this. I like the idea. The question is, should this be implemented in Spark or some other framework? I know that dbt has a fairly extensive way of testing your data , and making sure that you can enforce assumptions on t