While validating DSv2 transaction and changelog support that is coming in 4.2, we found a few items that must be fixed in order to make these features usable. - DSv2 transactions block dataframe caching in DELETE/UPDATE/MERGE, making the transactions unusable in all real connectors (Andreas to provide a JIRA) - Not having options while loading changelog API prevents consumption of the changelog API in Iceberg / Delta (SPARK-56961)
I am working on SPARK-56961 and should have a PR out tomorrow. I know there has been some progress on polishing the transaction API as well, so we should be able to complete it by the end of this week. нд, 17 трав. 2026 р. о 20:43 huaxin gao <[email protected]> пише: > Hi all, > > I plan to cut Spark 4.2.0 RC1 on May 20, assuming there are no outstanding > release blockers. > > If you have any fixes that must be included in 4.2.0, please make sure > they are merged/backported to branch-4.2 before then. If you are aware of > any release blockers, please reply with the JIRA/PR and current status. > > Thanks, > Huaxin >
