Re: [VOTE] Add Geometry and Geography types for V3

+1 On Sat, Feb 8, 2025 at 1:02 PM Fokko Driesprong wrote: > +1 > > Op za 8 feb 2025 om 08:08 schreef Péter Váry >: > >> +1 >> >> On Fri, Feb 7, 2025, 21:20 Kevin Liu wrote: >> >>> +1 (non-binding) >>> It's great to see support for more data types in both parquet and >>> Iceberg! >>> >>> Best,

Re: [VOTE] Simplify multi-arg table metadata

+1 On Mon, Feb 10, 2025 at 7:40 AM Péter Váry wrote: > +1 > > On Mon, Feb 10, 2025, 03:44 Manu Zhang wrote: > >> +1 (non-binding) >> >> On Mon, Feb 10, 2025 at 10:25 AM roryqi wrote: >> >>> +1 >>> >>> xianjin 于2025年2月10日周一 10:02写道： >>> +1 (non-binding) On Mon, Feb 10, 2025 at 2

Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release

That shouldn't be included. There is still active discussion on the PR, and it needs more work. The author also removed the milestone. The VOTE is out, so we can start verifying. Kind regards, Fokko Op ma 10 feb 2025 om 05:41 schreef Manu Zhang : > There's still https://github.com/apache/iceberg

Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release

https://github.com/apache/iceberg/issues/10745 should not be bound to 1.8.0 either? Regards, Manu On Mon, Feb 10, 2025 at 4:36 PM Fokko Driesprong wrote: > That shouldn't be included. There is still active discussion on the PR, > and it needs more work. The author also removed the milestone. Th

Re: [DISCUSS] Table name in table metadata

Hi Manu, I'm just brainstorming about how this addition could be problematic: Even though it's not recommended, it's feasible to register the same table in multiple catalogs. Different catalogs could use different names for the same underlying table, see register_table(name, metadata_location)

Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release

No, I don't think that's bound to a version anyway, as Amogh pointed out on the issue. I've removed it from the milestone. Thanks Kind regards, Fokko Op ma 10 feb 2025 om 11:20 schreef Manu Zhang : > https://github.com/apache/iceberg/issues/10745 should not be bound to > 1.8.0 either? > > Regard

Re: [VOTE] Simplify multi-arg table metadata

+1 On Mon, Feb 10, 2025 at 2:50 AM Eduard Tudenhöfner wrote: > +1 > > On Mon, Feb 10, 2025 at 7:40 AM Péter Váry > wrote: > >> +1 >> >> On Mon, Feb 10, 2025, 03:44 Manu Zhang wrote: >> >>> +1 (non-binding) >>> >>> On Mon, Feb 10, 2025 at 10:25 AM roryqi wrote: >>> +1 xianjin 于

Re: [VOTE] Simplify multi-arg table metadata

+1 On Mon, Feb 10, 2025 at 7:43 AM Russell Spitzer wrote: > +1 > > On Mon, Feb 10, 2025 at 2:50 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: > >> +1 >> >> On Mon, Feb 10, 2025 at 7:40 AM Péter Váry >> wrote: >> >>> +1 >>> >>> On Mon, Feb 10, 2025, 03:44 Manu Zhang wrote: >>>

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) On Mon, Feb 10, 2025 at 8:05 AM Honah J. wrote: > +1 > > On Mon, Feb 10, 2025 at 7:43 AM Russell Spitzer > wrote: > >> +1 >> >> On Mon, Feb 10, 2025 at 2:50 AM Eduard Tudenhöfner < >> etudenhoef...@apache.org> wrote: >> >>> +1 >>> >>> On Mon, Feb 10, 2025 at 7:40 AM Péter Váry

Re: [DISCUSS] Table name in table metadata

I see the table identifier (catalog.namespace.table), including the table name, as a catalog concept rather than a table property. You can register a table with the same location but different names, which makes sense from a catalog perspective. This also makes tables more portable when moving or

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding). Thanks, Steve Zhang > On Feb 9, 2025, at 1:01 AM, Fokko Driesprong wrote: > > (Second attempt, the cat ran over the keyboard) > > Hey everyone, > > After the positive responses on the devlist >

Re: [VOTE] Simplify multi-arg table metadata

+1 Yufei On Mon, Feb 10, 2025 at 9:48 AM Steve Zhang wrote: > +1 (non-binding). > > Thanks, > Steve Zhang > > > > On Feb 9, 2025, at 1:01 AM, Fokko Driesprong wrote: > > (Second attempt, the cat ran over the keyboard) > > Hey everyone, > > After the positive responses

Re: [VOTE] Add Geometry and Geography types for V3

Here is my +1 (binding) Thanks Szehon On Mon, Feb 10, 2025 at 12:47 AM Eduard Tudenhöfner < etudenhoef...@apache.org> wrote: > +1 > > On Sat, Feb 8, 2025 at 1:02 PM Fokko Driesprong wrote: > >> +1 >> >> Op za 8 feb 2025 om 08:08 schreef Péter Váry > >: >> >>> +1 >>> >>> On Fri, Feb 7, 2025, 21:

Re: [DISCUSS] Table name in table metadata

I don't think it is a good idea to add the table name to metadata because it can easily get stale and would be misleading. Table name is a catalog concern and we typically try to keep catalog concerns out of the table space. Instead, I'd suggest updating the error that your users see so that the er

Re: [VOTE] Release Apache Iceberg 1.8.0 RC0

+1 I verified signature, checksums, license, built and ran tests locally. Thanks for taking care of the release, Amogh! Thanks, Anurag On Sun, Feb 9, 2025 at 10:39 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > Hi Everyone, > > I propose that we release the following RC as the official Apache

Re: Table metadata swap not work for REST Catalog (#12134)

Yeah, it sounds like a "register table force" is the right concept here. I think we want to make sure that table updates remain change-based as the best practice in the REST API. But there are some irregular use cases that justify having some mechanism to completely replace the state (like push-bas

Re: Table metadata swap not work for REST Catalog (#12134)

Thank you Russell and Ryan. Let me start to work on a new API to support force table registration in catalog. Thanks, Steve Zhang > On Feb 10, 2025, at 4:29 PM, rdb...@gmail.com wrote: > > Yeah, it sounds like a "register table force" is the right concept here. I > think we want to make

Re: Table metadata swap not work for REST Catalog (#12134)

The push-based mirroring highlighted by Ryan is a popular use case. Polaris has already implemented notification APIs to address this, and there have been several Iceberg community discussions surrounding it. If I recall correctly, we generally agreed that the notification API is beneficial per las

Re: Table metadata swap not work for REST Catalog (#12134)

Hey Steve, I think the issue here is that you're using the commit api in table operations to perform a non-incremental/linear change to the metadata. The REST implementation is a little more strict in that it builds a set of updates based on the mutations made to the metadata and the commit proce

Re: Table metadata swap not work for REST Catalog (#12134)

I still would like a "register table" force" option On Mon, Feb 10, 2025 at 5:06 PM Steve Zhang wrote: > Thank you Dan for your detailed reply. Based on your explanation, do you > think it would be worthwhile to support non-linear or complete metadata > replacements in the REST implementation? I

[VOTE] Deprecate or remove distinct_count

This vote will be open for at least 72 hours. I propose that distinct_counts be either deprecated (#12182 ) or removed (#12183 ) from the spec. According to #767

[DISCUSS] Consolidate docs under Concepts and Project/Terms

Hi all, On the website, we have docs under Concepts with only Catalogs[1] and Project/Terms[2] serving similar purposes. Do you think it's a good idea to consolidate the two pages? For example, move all items under Terms to Concepts for better visibility. 1. https://iceberg.apache.org/concepts/ca

Re: Changelog scan for table with delete files

Hi Anton, Thank you for looking at https://github.com/apache/iceberg/pull/10935. I think we are in agreement on the behavior, but you have concerns about the performance of the scan, which I agree is justified. It has been some months now. Do you have any suggestions for improving the performance?

Re: [VOTE] Deprecate or remove distinct_count

Hi Jacob, Thanks for initiating the vote. Typically, we would first have a DISCUSSION thread to reach a consensus on the preferred option and then follow it up with a VOTE thread for confirmation. Maybe we can take this as a DISCUSSION thread? Best, Manu On Tue, Feb 11, 2025 at 7:20 AM Jacob M

Re: [VOTE] Add Geometry and Geography types for V3

This vote has passed with the following results: 9 +1 binding votes (Yufei, Ryan, Russell, Dan, Honah, Péter, Fokko, Eduard, Szehon) 10 +1 non-binding votes (Jia, Harsh, Huang-Hsiang, Steve, Denny, Gang, Jean-Baptiste, Aihua, Huaxin, Kevin ) I will merge the PR shortly. Thanks everyone for parti

Re: Table metadata swap not work for REST Catalog (#12134)

Thank you Dan for your detailed reply. Based on your explanation, do you think it would be worthwhile to support non-linear or complete metadata replacements in the REST implementation? I am happy to contribute but might need some guidance from the community on the best approach. For additional