Re: [DISCUSS] FileFormat API proposal

2025-03-21 Thread Renjie Liu
Hi, Peter: Thanks for the effort on this. *1. **File format filters* Do the filters include both filter expressions from both user query and delete filter? For filters from user query, I agree with you that we should keep the current behavior. For delete filters associated with data files, at

Re: [Discuss] Print un-pretty metadata JSON files without whitespace

2025-03-21 Thread Marc Cenac
> Perhaps this is a good default on the catalog side when creating new metadata json. +1 for this b/c I think it's an easy performance win for tables with large metadata. Is there any reason not to have write.metadata.compression-codec default to gzip? I'm curious if there was a reason it's curr

Re: [DISCUSS] Row lineage required for v3

2025-03-21 Thread Amogh Jahagirdar
I support enabling row lineage by default primarily because of the ecosystem benefit that enables engines to rely on lineage without requiring users to opt in explicitly. This should generally apply to most engines and integrations. However, as we know there are specific cases in the ecosystem—su

Re: [VOTE] Minor simplifications for Geo Spec

2025-03-21 Thread Russell Spitzer
+1 (bind On Fri, Mar 21, 2025 at 11:53 AM Steve Zhang wrote: > +1 (non-binding) > > Thanks, > Steve Zhang > > > > On Mar 18, 2025, at 6:29 PM, Gang Wu wrote: > > +1 (non-binding) > > >

Re: [VOTE] Minor simplifications for Geo Spec

2025-03-21 Thread Steve Zhang
+1 (non-binding) Thanks, Steve Zhang > On Mar 18, 2025, at 6:29 PM, Gang Wu wrote: > > +1 (non-binding)

Re: [DISCUSS] Row lineage required for v3

2025-03-21 Thread Ryan Blue
For streaming applications (Flink, Kafka Connect) this is a non-trivial task. The engine either has to keep everything in memory or do continuous lookups. Sorry, I should be more clear. I was referring to the work of propagating row ID and last updated sequence number, *assuming that the engine is

Re: [VOTE] Minor simplifications for Geo Spec

2025-03-21 Thread Amogh Jahagirdar
+1 On Thu, Mar 20, 2025 at 12:50 PM Daniel Weeks wrote: > +1 > > On Wed, Mar 19, 2025 at 1:50 PM Jean-Baptiste Onofré > wrote: > >> +1 (non binding) >> >> Regards >> JB >> >> On Wed, Mar 19, 2025 at 1:01 AM Szehon Ho >> wrote: >> > >> > Hi everyone, >> > >> > While working on the reference imp