Sorry I didn't get to reply here, I think the fix Ajantha is contributing is extremely important but probably out of scope for a patch release. Because it will take a bit of manual intervention to fix after jumping to the next version I think we should save this for 1.10.0 which also should come out pretty soon.
On Tue, Jun 17, 2025 at 1:16 PM Prashant Singh <prashant010...@gmail.com> wrote: > Thank you for confirming over slack Ajantha, I also double checked with > Russell offline, this seems to be a behaviour change which can't go in a > patch release, maybe 1.10 then. So I think we should be good for now. > That being said, I will start working on getting a RC out with just this > commit > <https://github.com/singhpk234/iceberg/commit/14ba08e672df5061f8ac0978ea81f10535da2246> > on top of 1.9.1, and will start a thread for that accordingly ! > > Best, > Prashant Singh > > On Tue, Jun 17, 2025 at 8:44 AM Prashant Singh <prashant010...@gmail.com> > wrote: > >> Hey Ajantha, >> >> I was going to wait for 2-3 days before cutting an RC anyway :), to see >> if anyone has an objection or some more *critical* changes to get in. >> Thank you for the heads up ! >> >> Best, >> Prashant Singh >> >> >> >> >> On Mon, Jun 16, 2025 at 11:02 AM Ajantha Bhat <ajanthab...@gmail.com> >> wrote: >> >>> I have a PR that needs to be included for 1.9.2 as well! >>> >>> I was about to start a release discussion for 1.9.2 tomorrow. >>> >>> It is from an oversight during partition stats refactoring (in 1.9.0). >>> We missed that field ids are tracked in spec during refactoring! Because of >>> it, schema field ids are not as per partition stats spec. Solution is to >>> implicitly recompute stats when old stats are detected with invalid field >>> id. Including the fix in 1.9.2 can reduce the issue impact as new stats >>> will have valid schema ids and also handle corrupted stats. Feature is not >>> widely integrated yet and engine integrations are in progress for 1.10.0. >>> But we still want to reduce the impact of the issue. I have chatted with >>> Peter and Russell about this. >>> >>> PR will be shared tomorrow. >>> >>> Just giving heads up to wait a day or two for a release cut. >>> >>> - Ajantha >>> >>> On Mon, Jun 16, 2025 at 11:25 PM Steven Wu <stevenz...@gmail.com> wrote: >>> >>>> +1 for a 1.9.2 release >>>> >>>> On Mon, Jun 16, 2025 at 10:53 AM Prashant Singh < >>>> prashant010...@gmail.com> wrote: >>>> >>>>> Hey Kevin, >>>>> This goes well before 1.8, if you will see the issue that my PR refers >>>>> to is reported from iceberg 1.7, It has been there since the beginning of >>>>> the IRC client. >>>>> We were having similar debates on if we should patch all the releases, >>>>> but I think this requires more wider discussions, but 1.9.2 sounds like >>>>> the >>>>> immediate next steps ! >>>>> >>>>> Best, >>>>> Prashant Singh >>>>> >>>>> On Mon, Jun 16, 2025 at 10:42 AM Kevin Liu <kevinjq...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi Prashant, >>>>>> >>>>>> This sounds like a good reason to do a patch release. I'm +1 >>>>>> >>>>>> Do you know if this is also affecting other minor versions? Do we >>>>>> also need to patch 1.8.x? >>>>>> >>>>>> Best, >>>>>> Kevin Liu >>>>>> >>>>>> On Mon, Jun 16, 2025 at 10:30 AM Prashant Singh < >>>>>> prashant010...@gmail.com> wrote: >>>>>> >>>>>>> Hey all, >>>>>>> >>>>>>> A couple of users have recently reported a *critical table >>>>>>> corruption issue* when using the Iceberg REST catalog client. This >>>>>>> issue has existed from the beginning and can result in *committed >>>>>>> snapshot data being incorrectly deleted*, effectively rendering the >>>>>>> table unusable (i.e., corrupted). The root cause is *self-conflict >>>>>>> during internal HTTP client retries*. >>>>>>> >>>>>>> More context can be found here: >>>>>>> [1] https://github.com/apache/iceberg/pull/12818#issue-3000325526 >>>>>>> [2] >>>>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1747992294134219 >>>>>>> >>>>>>> We’ve seen this issue hit by users in the wider community and also >>>>>>> internally on our end. I’ve submitted a fix here (now merged) : >>>>>>> https://github.com/apache/iceberg/pull/12818 >>>>>>> >>>>>>> Given the *severity* and the *broad version impact*, I’d like to >>>>>>> propose cutting a *1.9.2 patch release* to include this fix. >>>>>>> >>>>>>> I'm also happy to *volunteer as the release manager* for this >>>>>>> release. >>>>>>> >>>>>>> Looking forward to hearing the community's thoughts. >>>>>>> >>>>>>> Best regards, >>>>>>> Prashant Singh >>>>>>> >>>>>>
