Re: [DISCUSS] Support keeping at most N snapshots

2025-01-17 Thread Manu Zhang
The intention is to set an upper limit for table size while keeping as much snapshots as possible. Setting max-snapshot-age-ms to a small value will lose history for some tables while setting min-snapshots-to-keep to a medium value will keep too much history for others. Lewis, William 于2025年1月18日

Re: [DISCUSS] Support keeping at most N snapshots

2025-01-17 Thread Lewis, William
To clarify, the intention is that if max-snapshots-to-keep is set, then snapshots will be expired even if they are younger than max-snapshot-age-ms? Does this solve a problem that isn’t solved by setting max-snapshot-age-ms to a small value and setting min-snapshots-to-keep to your desired value

Re: [Discuss][Vote] Spec Change - Add optional field added-rows to Snapshot for Row Lineage

2025-01-17 Thread Fokko Driesprong
+0, as I agree with Amogh, I think it would fit nicely with Honah's work of formalizing the properties. Kind regards, Fokko Op vr 17 jan 2025 om 08:55 schreef Honah J. : > +1 > > Best, > Honah > > On Thu, Jan 16, 2025 at 22:54 Manish Malhotra < > manish.malhotra.w...@gmail.com> wrote: > >> +1,

Re: [Discuss][Vote] Spec Change - Add optional field added-rows to Snapshot for Row Lineage

2025-01-17 Thread Russell Spitzer
Thanks everyone, especially Amogh and Fokko for providing other views on this change. I think your viewpoints are definitely valid although I think currently we should be working on moving things away from Snapshot Summary if at all possible. I think it's better if the information there is never cr

Re: [Discuss][Vote] Spec Change - Add optional field added-rows to Snapshot for Row Lineage

2025-01-17 Thread Jean-Baptiste Onofré
+1 (non binding) Regards JB On Wed, Jan 15, 2025 at 5:59 PM Russell Spitzer wrote: > > Hi Everyone! > > PR: https://github.com/apache/iceberg/pull/11976/files > > Split out from #11948 > > Working on the row-lineage implementation made it clear that we needed a way > to get information from the