Re: [ANNOUNCE] Apache Iceberg release 1.7.1

2024-12-09 Thread Eduard Tudenhöfner
Thanks Bryan and everyone else for making this release happen. On Tue, Dec 10, 2024 at 12:27 AM Yuya Ebihara wrote: > Thank you Brian! Trino project had waited for 1.7.1 that fixes namespace > regression. >

Re: [DISCUSS] Apache Iceberg Summit 2025 - Selection Committee

2024-12-09 Thread Ashvin A
Hi JB, Thanks for driving this. I'd love to volunteer. Best, Ashvin On Mon, Dec 9, 2024 at 12:06 PM Aihua Xu wrote: > Hi JB, > > I would love to help. > > Thanks, > Aihua > > On Tue, Dec 3, 2024 at 4:37 AM Anurag Mantripragada > wrote: > >> Thanks for the email. I’m happy to help as well. >>

Re: [VOTE] Add Variant type to Iceberg Spec

2024-12-09 Thread Aihua Xu
Thank you all for the discussion. Micah raised a valid concern about including a specification that has not yet been finalized in Parquet. As we discussed earlier, the community has shown interest in introducing the basic variant type and shredding in V3. >From my perspective, while the change is

Re: [ANNOUNCE] Apache Iceberg release 1.7.1

2024-12-09 Thread Yuya Ebihara
Thank you Brian! Trino project had waited for 1.7.1 that fixes namespace regression.

Re: REST catalog high availability

2024-12-09 Thread Yufei Gu
Load balancing operates at a different layer than APIs, with various implementations available, such as etcd and Zookeeper. I’d prefer to avoid introducing additional complexity at the web service API level. Yufei On Mon, Dec 9, 2024 at 8:35 AM Jean-Baptiste Onofré wrote: > Hi Vladimir, > > As

Re: [ANNOUNCE] Apache Iceberg release 1.7.1

2024-12-09 Thread Yufei Gu
Thanks a lot, Byran! It's great to have multiple things fixed in the new version. It enables upgrades from varieties of downstream projects! Yufei On Mon, Dec 9, 2024 at 12:37 PM Russell Spitzer wrote: > Thanks so much Bryan! Great work getting this out! > > On Mon, Dec 9, 2024 at 2:28 PM Brya

Re: [ANNOUNCE] Apache Iceberg release 1.7.1

2024-12-09 Thread Russell Spitzer
Thanks so much Bryan! Great work getting this out! On Mon, Dec 9, 2024 at 2:28 PM Bryan Keller wrote: > I'm pleased to announce the release of Apache Iceberg 1.7.1! > > Apache Iceberg is an open table format for huge analytic datasets. Iceberg > delivers high query performance for tables with te

[ANNOUNCE] Apache Iceberg release 1.7.1

2024-12-09 Thread Bryan Keller
I'm pleased to announce the release of Apache Iceberg 1.7.1! Apache Iceberg is an open table format for huge analytic datasets. Iceberg delivers high query performance for tables with tens of petabytes of data, along with atomic commits, concurrent writes, and SQL-compatible table evolution. This

Re: [DISCUSS] Apache Iceberg Summit 2025 - Selection Committee

2024-12-09 Thread Aihua Xu
Hi JB, I would love to help. Thanks, Aihua On Tue, Dec 3, 2024 at 4:37 AM Anurag Mantripragada wrote: > Thanks for the email. I’m happy to help as well. > > > Anurag Mantripragada > > On Dec 1, 2024, at 7:27 PM, Nick Riasanovsky wrote: > > Happy to volunteer as well. > > - Nick Riasanovsky >

Re: REST catalog high availability

2024-12-09 Thread Jean-Baptiste Onofré
Hi Vladimir, As you said, today, it's possible to use a LB in front of multiple instances (using nginx, ELB, ...). I think it's pretty easy to setup and at "infrastructure" level. As it's possible to plug the HTTP5 client in Iceberg REST client, I think it's possible to inject PoolingHttpClientCo

Re: REST catalog high availability

2024-12-09 Thread Samrose Ahmed
In my opinion, this is unnecessary, it's well solved by load balancers/proxies. On Mon, Dec 9, 2024, 8:12 AM Vladimir Ozerov wrote: > Hi, > > Catalog is a critical part of Iceberg infrastructure and may require > highly available setup. In similar services (e.g., HMS, etc) this is often > done a

REST catalog high availability

2024-12-09 Thread Vladimir Ozerov
Hi, Catalog is a critical part of Iceberg infrastructure and may require highly available setup. In similar services (e.g., HMS, etc) this is often done as follows: 1. Start several service instances 2. Decide which one is coordinator via etcd, Zookeper, Ratis, etc 3. Expose HA endpoint

Re: [DISCUSS] Relocate Parquet to Iceberg Core

2024-12-09 Thread Ajantha Bhat
Thanks Dan for the reply. This is also a good time to consider adding a native parquet read/write > path for use in core as the generic path in 'iceberg-data' isn't ideal. > Parquet metadata has been brought up in relation to improving stats > handling (allowing tracking of more column stats witho

Re: [PROPOSAL] Create Iceberg DockerHub repository

2024-12-09 Thread Jean-Baptiste Onofré
Hi Piotr That's a good point. As DockerHub is managed by The ASF, I think it's worth it to have docker images hosted there at least. That said, I don't see a problem with publishing on GH Packages. Regards JB On Thu, Dec 5, 2024 at 2:57 PM Piotr Findeisen wrote: > > Hi, > > Sorry for coming la