Re: Stableness of V2 Spec/API

I just commented on #2303. I think we should get that fixed fairly soon -- at least an interim fix to ensure that compaction correctly catches the problem and fails. The plan for the long-term fix looks good to me as well. On Mon, May 17, 2021 at 7:17 PM OpenInx wrote: > The PR-2303 defines how

Re: Stableness of V2 Spec/API

The PR-2303 defines how the batch job does the compaction work, the PR-2308 decides what's the behavior that compaction txn and row-delta txn commit at the same time. They should n't block each other, but we will need to resolve both of them. On Tue, May 18, 2021 at 9:36 AM Huadong Liu wrot

Re: Stableness of V2 Spec/API

Thanks. Compaction is https://github.com/apache/iceberg/pull/2303 and it is currently blocked by https://github.com/apache/iceberg/issues/2308? On Mon, May 17, 2021 at 6:17 PM OpenInx wrote: > Hi Huadong > > From the perspective of iceberg developers, we don't expose the format v2 > to end users

Re: Stableness of V2 Spec/API

Hi Huadong >From the perspective of iceberg developers, we don't expose the format v2 to end users because we think there is still other work that needs to be done. As you can see there are still some unfinished issues from your link. As for whether v2 will cause data loss, from my perspective as

Stableness of V2 Spec/API

Hi iceberg-dev, I tried v2 row-level deletion by committing equality delete files after *upgradeToFormatVersion(2)*. It worked well. I know that Spark actions to compact delete files and data files etc. are in progress. I currently use the JAVA API t