Re: [VOTE] Deletion Vectors in V3

+1 I had to read up a bit, thanks for driving this Anton. Kind regards, Fokko Op do 31 okt 2024 om 07:53 schreef Piotr Findeisen < piotr.findei...@gmail.com>: > Thank you Anton, > > +1 (non-binding) > > > > On Thu, 31 Oct 2024 at 05:07, John Zhuge wrote: > >> +1 (non-binding) >> >> On Wed, Oct

Re: [VOTE] Deletion Vectors in V3

Thank you Anton, +1 (non-binding) On Thu, 31 Oct 2024 at 05:07, John Zhuge wrote: > +1 (non-binding) > > On Wed, Oct 30, 2024 at 1:56 PM Anton Okolnychyi > wrote: > >> +1 (binding) >> >> - Anton >> >> ср, 30 жовт. 2024 р. о 21:32 Amogh Jahagirdar <2am...@gmail.com> пише: >> >>> +1 (binding)

Re: [VOTE] Deletion Vectors in V3

+1 (non-binding) On Wed, Oct 30, 2024 at 1:56 PM Anton Okolnychyi wrote: > +1 (binding) > > - Anton > > ср, 30 жовт. 2024 р. о 21:32 Amogh Jahagirdar <2am...@gmail.com> пише: > >> +1 (binding) >> >> Thanks, >> Amogh Jahagirdar >> >> On Wed, Oct 30, 2024 at 1:28 PM Bryan Keller wrote: >> >>> +1!

Re: [DISCUSS] - Deprecate Equality Deletes

I think Apache Paimon could point us in the direction of supporting streaming upserts use cases. We are already working on some of the building blocks like deletion vectors and Flink compaction. +1 to the proposal since users are not recommended to use equality deletes for streaming upserts anyway

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

Technically those binaries are not the release. Only the source is the release and those binaries are provided as a convenience, although that said we should probably fix it. On Wed, Oct 30, 2024 at 9:26 PM Ajantha Bhat wrote: > Hi Russell, > Thanks for working on the release. > > I just found t

Re: [DISCUSS] - Deprecate Equality Deletes

I have a very basic question. Is there already an alternative to equality deletes for flink? If we deprecate the feature, is because there’s already an alternative correct ? On Wed, Oct 30, 2024 at 7:50 PM Manu Zhang wrote: > I think Apache Paimon could point us in the direction of supporting >

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

Hi Russell, Thanks for working on the release. I just found the issue[1] that none of the Iceberg runtime jars are including the LICENSE and NOTICE from the respective module directory. It always only packs the LICENSE and NOTICE from the root directory. This is observed even in the old releases l

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

Thanks Russell for making the release. Not a blocker but I really wish this doc change[1] can be included into 1.7.0. [1] https://github.com/apache/iceberg/pull/11417 On Thu, Oct 31, 2024 at 7:00 AM Russell Spitzer wrote: > Convenience binary artifacts are staged on Nexus. The Maven repository

Re: [DISCUSS] - Deprecate Equality Deletes

Equality deletes aren't only written from Flink; Iceberg Kafka Connect (Tabular’s version) also writes equality deletes for upserts. Writers write out reference to what values are deleted (in a partition or > globally). There can be an unlimited number of equality deletes and they > all must be ch

Re: [DISCUSS] - Deprecate Equality Deletes

Thanks Russell for bringing this up! +1 on deprecating equality deletes. IMHO, this is something that should reside only in the ingestion engine. Best, Gang On Thu, Oct 31, 2024 at 5:07 AM Russell Spitzer wrote: > Background: > > 1) Position Deletes > > > Writers determine what rows are delet

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

Convenience binary artifacts are staged on Nexus. The Maven repository URL is: https://repository.apache.org/content/repositories/orgapacheiceberg-1175/ On Wed, Oct 30, 2024 at 5:31 PM Russell Spitzer wrote: > Sorry forgot binary, doing that now > > On Wed, Oct 30, 2024 at 5:06 PM Russell Spitze

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

Sorry forgot binary, doing that now On Wed, Oct 30, 2024 at 5:06 PM Russell Spitzer wrote: > Hey Y'all, > > I propose that we release the following RC as the official Apache Iceberg > 1.7.0 release. > > The commit ID is 91e04c9c88b63dc01d6c8e69dfdc8cd27ee811cc > * This corresponds to the tag: ap

[VOTE] Release Apache Iceberg 1.7.0 RC0

Hey Y'all, I propose that we release the following RC as the official Apache Iceberg 1.7.0 release. The commit ID is 91e04c9c88b63dc01d6c8e69dfdc8cd27ee811cc * This corresponds to the tag: apache-iceberg-1.7.0-rc0 * https://github.com/apache/iceberg/commits/apache-iceberg-1.7.0-rc0 * https://gith

[DISCUSS] - Deprecate Equality Deletes

Background: 1) Position Deletes Writers determine what rows are deleted and mark them in a 1 for 1 representation. With delete vectors this means every data file has at most 1 delete vector that it is read in conjunction with to excise deleted rows. Reader overhead is more or less constant and i

Re: [VOTE] Deletion Vectors in V3

+1 (binding) - Anton ср, 30 жовт. 2024 р. о 21:32 Amogh Jahagirdar <2am...@gmail.com> пише: > +1 (binding) > > Thanks, > Amogh Jahagirdar > > On Wed, Oct 30, 2024 at 1:28 PM Bryan Keller wrote: > >> +1! >> >> On Oct 30, 2024, at 11:03 AM, Daniel Weeks wrote: >> >> +1 (binding) >> >> -Dan >> >>

Re: [VOTE] Deletion Vectors in V3

+1 (binding) Thanks, Amogh Jahagirdar On Wed, Oct 30, 2024 at 1:28 PM Bryan Keller wrote: > +1! > > On Oct 30, 2024, at 11:03 AM, Daniel Weeks wrote: > > +1 (binding) > > -Dan > > On Wed, Oct 30, 2024 at 10:51 AM Prashant Singh > wrote: > >> +1 (non-binding) >> >> Thanks, >> Prashant >> >> On

Re: [VOTE] Deletion Vectors in V3

+1! > On Oct 30, 2024, at 11:03 AM, Daniel Weeks wrote: > > +1 (binding) > > -Dan > > On Wed, Oct 30, 2024 at 10:51 AM Prashant Singh > wrote: >> +1 (non-binding) >> >> Thanks, >> Prashant >> >> On Wed, Oct 30, 2024 at 10:16 AM Russell Spitzer >

Re: [VOTE] Deletion Vectors in V3

+1 - My comments are clear on the other thread for future proposals On Wed, Oct 30, 2024 at 11:18 AM Szehon Ho wrote: > -0 > > Great work and exciting functionality, but transferring concerns from the > other thread about the decision. > > Thanks > Szehon > > On Wed, Oct 30, 2024 at 9:12 AM Ste

Re: [VOTE] Deletion Vectors in V3

+1 (binding) -Dan On Wed, Oct 30, 2024 at 10:51 AM Prashant Singh wrote: > +1 (non-binding) > > Thanks, > Prashant > > On Wed, Oct 30, 2024 at 10:16 AM Russell Spitzer < > russell.spit...@gmail.com> wrote: > >> +1 - My comments are clear on the other thread for future proposals >> >> On Wed, O

Re: [VOTE] Deletion Vectors in V3

+1 (non-binding) Thanks, Prashant On Wed, Oct 30, 2024 at 10:16 AM Russell Spitzer wrote: > +1 - My comments are clear on the other thread for future proposals > > On Wed, Oct 30, 2024 at 11:18 AM Szehon Ho > wrote: > >> -0 >> >> Great work and exciting functionality, but transferring concern

Re: [VOTE] Deletion Vectors in V3

-0 Great work and exciting functionality, but transferring concerns from the other thread about the decision. Thanks Szehon On Wed, Oct 30, 2024 at 9:12 AM Steven Wu wrote: > +1 > > On Wed, Oct 30, 2024 at 1:07 AM xianjin wrote: > >> +1 (non binding) >> >> On Wed, Oct 30, 2024 at 2:28 PM Jean

Re: [VOTE] Deletion Vectors in V3

+1 On Wed, Oct 30, 2024 at 1:07 AM xianjin wrote: > +1 (non binding) > > On Wed, Oct 30, 2024 at 2:28 PM Jean-Baptiste Onofré > wrote: > >> +1 (non binding) >> >> Regards >> JB >> >> On Tue, Oct 29, 2024 at 10:45 PM Anton Okolnychyi >> wrote: >> > >> > Hi folks, >> > >> > We have been discussi

Re: [VOTE] Deletion Vectors in V3

+1 (non-binding) ~ Anurag Mantripragada > On Oct 30, 2024, at 1:07 AM, xianjin wrote: > > +1 (non binding) > > On Wed, Oct 30, 2024 at 2:28 PM Jean-Baptiste Onofré > wrote: >> +1 (non binding) >> >> Regards >> JB >> >> On Tue, Oct 29, 2024 at 10:45 PM Anton O

REST catalog removes void transform

Hi, When a user creates a table with void() transform on a single column, REST catalogs appears to ignore this, and ends up with a table with no partitioning information. The relevant code part is in RESTSessionCatalog.createChanges: PartitionSpec spec = meta.spec(); if (spec != null && spec.isP

Re: [VOTE] Deletion Vectors in V3

+1 (non binding) On Wed, Oct 30, 2024 at 2:28 PM Jean-Baptiste Onofré wrote: > +1 (non binding) > > Regards > JB > > On Tue, Oct 29, 2024 at 10:45 PM Anton Okolnychyi > wrote: > > > > Hi folks, > > > > We have been discussing the new layout for position deletes in V3 for a > while now. It seems

Re: [VOTE] Deletion Vectors in V3

+1 On Wed, Oct 30, 2024 at 7:28 AM Jean-Baptiste Onofré wrote: > +1 (non binding) > > Regards > JB > > On Tue, Oct 29, 2024 at 10:45 PM Anton Okolnychyi > wrote: > > > > Hi folks, > > > > We have been discussing the new layout for position deletes in V3 for a > while now. It seems the community