I have a working end-to-end solution locally using some of the upcoming features in Spark 3.2 and a more elaborate version of what is available in this Spark PR: https://github.com/apache/spark/pull/33008 <https://github.com/apache/spark/pull/33008>
More details on the proposed Spark APIs are in this SPIP (slightly outdated): https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60/edit <https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60/edit> At this point, I am submitting Iceberg PRs one by one and would appreciate reviews/comments. Here is a list of current Iceberg PRs that are relevant: - Fix commit validation for delete files (#3069) - New writer interfaces (probably the largest part here) (#2945) - Support committing delete files to multiple specs (#2985) - Support for _spec_id and _partition metadata columns (#2984) Apart from that, we will need to implement an action to rewrite delete files. There was a PR but did not have a chance to take a look at it yet. - Anton > On 15 Sep 2021, at 07:46, Aman Rawat <aman.ra...@salesforce.com.INVALID> > wrote: > > Hey team, > > We are trying to implement Spark support for Row Level Deletes for iceberg. > Can you please throw some light on - where this work stream is & how can we > help. > > Regards,
