Thanks a lot Anton. This is really helpful. We will read through this and check out the current PRs to get up to speed here.
Best Regards, Aman. On Wed, Sep 15, 2021 at 9:53 PM Anton Okolnychyi <aokolnyc...@apple.com.invalid> wrote: > Here is the action PR I mentioned: > https://github.com/apache/iceberg/pull/2841 > <https://urldefense.com/v3/__https://github.com/apache/iceberg/pull/2841__;!!DCbAVzZNrAf4!X-f0W0n0FxCNt_L8RSC7EzSaP-KtpJSC9SHC8kvP1lMEMDgkp4Ym3OiJliG6IC-OLn0$> > > - Anton > > On 15 Sep 2021, at 09:14, Anton Okolnychyi <aokolnyc...@apple.com.INVALID> > wrote: > > I have a working end-to-end solution locally using some of the upcoming > features in Spark 3.2 and a more elaborate version of what is available in > this Spark PR: > https://github.com/apache/spark/pull/33008 > <https://urldefense.com/v3/__https://github.com/apache/spark/pull/33008__;!!DCbAVzZNrAf4!X-f0W0n0FxCNt_L8RSC7EzSaP-KtpJSC9SHC8kvP1lMEMDgkp4Ym3OiJliG6VdcSeJM$> > > More details on the proposed Spark APIs are in this SPIP (slightly > outdated): > > https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60/edit > <https://urldefense.com/v3/__https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60/edit__;!!DCbAVzZNrAf4!X-f0W0n0FxCNt_L8RSC7EzSaP-KtpJSC9SHC8kvP1lMEMDgkp4Ym3OiJliG6KKKuzZk$> > > At this point, I am submitting Iceberg PRs one by one and would appreciate > reviews/comments. > > Here is a list of current Iceberg PRs that are relevant: > > - Fix commit validation for delete files (#3069) > - New writer interfaces (probably the largest part here) (#2945) > - Support committing delete files to multiple specs (#2985) > - Support for _spec_id and _partition metadata columns (#2984) > > Apart from that, we will need to implement an action to rewrite delete > files. There was a PR but did not have a chance to take a look at it yet. > > - Anton > > > On 15 Sep 2021, at 07:46, Aman Rawat <aman.ra...@salesforce.com.INVALID> > wrote: > > Hey team, > > We are trying to implement Spark support for Row Level Deletes for > iceberg. Can you please throw some light on - where this work stream is & > how can we help. > > Regards, > > > >
