Pucheng, I am not sure about others. At least I had some hard time understanding what the problem/proposal is. What is "cherrypick static partition overwrite"?
Thanks, Steven On Thu, May 30, 2024 at 11:59 AM Pucheng Yang <py...@pinterest.com.invalid> wrote: > Hi community, > > I would like to follow up on this proposal and would like to check if > anyone has concerns about the proposed implementation from a high level > perspective? > > Thank you very much > > Best, > Pucheng > > On Tue, May 28, 2024 at 9:02 PM Pucheng Yang <py...@pinterest.com> wrote: > >> Hi community, >> >> My client is looking for the support of cherrypick static >> partition overwrite. Based on my understanding, the reason we can not do it >> is because we do not preserve static overwrite filters. >> >> I would like to make a proposal to support cherrypick static overwrite: >> 1. We will allow users to opt in preserving static overwrite filters >> during snapshot summary >> 2. When the snapshot is being cherry picked, we will first delete from >> the current snapshot using the static overwrite fitler preserved in the >> snapshot, then we will redo the add files operation. >> >> Please share if you have any thoughts or comments on this, if we agree, I >> can open up a PR for this. >> >> Best, >> Pucheng >> >
