Re: [DISCUSS] Describing REST Server capabilities

We had a separate discussion with Dan on the *oauth2* flag last week and came to the same conclusion that removing the *oauth2* capability is probably the best for now. This is mainly because we can't really act on the *oauth2* capability right now, because the */tokens* endpoint is called before w

Re: [Discussion] Apache Iceberg Guidelines for Committership and PMC Membership

Thanks Jack for adding this doc. I just had one minor comment about the duration of "sustained" contributions but overall the doc LGTM Eduard On Tue, Jun 25, 2024 at 8:11 PM Jack Ye wrote: > Hi everyone, > > Here is a draft proposal for the guidelines for committership and PMC > membership: >

Re: [DISCUSS] Describing REST Server capabilities

gt; On Mon, Jun 24, 2024 at 1:42 PM Micah Kornfield < >>> emkornfi...@gmail.com> >>> > >>> wrote: >>> > >>> >>> > >>>> I don't have strong opinions either way here, just thought it was >>> wo

Re: [DISCUSS] Describing REST Server capabilities

I couldn't make it to the catalog sync meeting yesterday but I watched the recording today (thanks for providing that). The missing piece is how (new, capabilities-aware) clients handle the case > when a service does _not_ return the capabilities field (absent). My > proposal would be that a clien

Re: [DISCUSS] Describing REST Server capabilities

t;> capability. (1) seems to be cleaner, but has some overhead in provisioning >> a new endpoint compared to (2). >> >> Do you see another way to do this leveraging the table-spec version? >> >> -Jack >> >> >> >> >> >> On Tue, Jul 2

[DISCUSS] Fix property names in REST spec for statistics / partition statistics

Hey everyone, I've opened #10662 to fix property names for statistics / partition statistics in the REST spec. I can start a separate VOTE thread if there is agreement around the proposed Spec change. Thanks Eduard

Re: [DISCUSS] Describing REST Server capabilities

> that I proposed before that the capabilities can be > overridden/provided by server implementation. Else, I'm afraid we > won't be flexible enough or always behind the implementation (if an > implementation wants to add "my-foo-cap"). > > Regards > JB > &

Re: [DISCUSS] Enable the discussion tab for iceberg github repos

I think GH discussions would be great to have on the Iceberg repo(s), so +1 from my side on this. Eduard On Tue, Jul 9, 2024 at 8:14 AM Renjie Liu wrote: > Hi: > > It's also possible to create a user mailing list if it helps. > > > I'm neutral to this option. Seems we are actually missing the u

[VOTE] Fix property names in REST spec for statistics / partition statistics

Hey everyone, I propose to fix the property names in the REST spec for statistics / partition statistics so that they are properly aligned with the table spec and the i

Re: Building with JDK 21

I'm generally in favor of dropping JDK 8 support and moving to JDK 21. The one blocker that is currently preventing us from doing that is Hive. What about deprecating the Hive stuff e.g. with 1.6.0 and dropping it with 1.7.0 and moving to the newer JDK in 1.7.0? Iceberg 1.6.0 would then be the las

Re: Building with JDK 21

> > > It has a caveat (we can't run formatter on 21 and 8, and we need to choose >> one). > > > Would it format differently? I would go for 21 since that's the path > forward, but I'm also fine with JB's suggestion 👍 > >> >> Yeah it would format differently, because the underlying *google-java-form

Re: [DISCUSS] Describing REST Server capabilities

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/501> (this >>>>> implies that the server impl would e.g. return a *404* in such a >>>>> case). >>>>> >>>>> My reading on the Mozilla page makes me think that it is phrased too

Re: [VOTE] spec: remove the JSON spec for content file and file scan task sections

+1 (non-binding) On Thu, Jul 11, 2024 at 8:29 AM Ajantha Bhat wrote: > +1 (non-binding) > > - Ajantha > > On Thu, Jul 11, 2024 at 11:02 AM Jean-Baptiste Onofré > wrote: > >> +1 (non binding) >> >> Regards >> JB >> >> On Thu, Jul 11, 2024 at 12:50 AM Steven Wu wrote: >> > >> > Following the lat

Re: [DISCUSS] Describing REST Server capabilities

t moves the client operations > into a mode where the server has more control (over having longer term > client-side credentials), so it looks like a reasonable mode to support > from the security perspective. > > Let's keep that capability flag. > > Cheers, > Dmitri. &

Re: [DISCUSS] Describing REST Server capabilities

y to per-endpoint versioning? >> Isn't it also nice to just fail at client side instead of calling server >> and getting a "generic 501"? >> >> -Jack >> >> >> >> >> >> >> >> On Thu, Jul 11, 2024 at 9:51 AM J

Re: [DISCUSS] Describing REST Server capabilities

t;> Isn't it also nice to just fail at client side instead of calling server >> and getting a "generic 501"? >> >> -Jack >> >> >> >> >> >> >> >> On Thu, Jul 11, 2024 at 9:51 AM Jack Ye wrote: >> >>>

Re: [VOTE] Fix property names in REST spec for statistics / partition statistics

The vote passed with *5 binding +1* votes and *10 non-binding +1* votes. Thanks everyone, I'll get the PR merged. On Fri, Jul 12, 2024 at 4:10 AM Honah J. wrote: > +1 (non-binding) > > Best regards, > Honah > > On Wed, Jul 10, 2024 at 1:19 PM Péter Váry > wrote: > >> +1 (non-binding - at least

Re: [DISCUSS] Describing REST Server capabilities

;tables > -> default capability in case the `capabilities` property doesn't exist or > is empty in the response" - meaning: the server would _only_ support > tables. This phrase in the spec proposal effectively removes the view > functionality from all currently existing Icebe

Re: [DISCUSS] Describing REST Server capabilities

pec v2 isn't far away either. > > Adding table-spec + view-spec capabilities now saves a lot of headaches > for Iceberg users in the near future. > > > On 15.07.24 11:27, Eduard Tudenhöfner wrote: > > I would suggest adding *table-spec / view-spec / udf-spec *capabiliti

Re: [VOTE] Release Apache Iceberg 1.6.0 RC0

+1 (non-binding) - verified signatures / checksums - ran license check - ran local tests - ran Spark integration tests Thanks JB for running the release. Eduard On Mon, Jul 15, 2024 at 1:50 PM Fokko Driesprong wrote: > Thanks JB for running the release! > > +1 (binding) > > - Chec

Re: [DISCUSS] Describing REST Server capabilities

ctly, declaring server > capabilities is an optimization to allow more efficient / less error-prone > client operation. I do not think it should impose additional functional / > interoperability requirements on servers. > > Cheers, > Dmitri. > > On Mon, Jul 15, 2024 at 1

Re: [DISCUSS] Deprecate HadoopTableOperations, move to tests in 2.0

+1 on deprecating now and removing them from the codebase with Iceberg 2.0 On Thu, Jul 18, 2024 at 10:40 AM Ajantha Bhat wrote: > +1 on deprecating the `File System Tables` from spec and `HadoopCatalog`, > `HadoopTableOperations` in code for now > and removing them permanently during 2.0 release

Re: [VOTE] Release Apache Iceberg 1.6.0 RC1

+1 (non-binding) - verified signatures / checksums / licenses - compiled and ran all tests using JDK17 Eduard On Mon, Jul 22, 2024 at 9:43 AM Fokko Driesprong wrote: > +1 (binding) > > - Validated checksums and signatures > - Checked licenses > - Compiled and ran the tests locally using

Re: [ANNOUNCE] Welcoming new committers and PMC members

Congrats everyone, it's amazing to see such great people contributing and improving the Iceberg community. On Wed, Jul 24, 2024 at 8:04 AM Honah J. wrote: > Thank you all! Congratulations to Kevin, Piotr, Sung, Xuanwo, and Renjie! > > It’s truly an honor to contribute to the project and witnes

Re: Meeting time for catalog community sync

+1 on having the catalog syncs on the Wednesdays when there's no Iceberg sync. On Thu, Jul 25, 2024 at 8:41 AM Robert Stupp wrote: > Prefer to keep Monday > > On 24.07.24 21:59, Jack Ye wrote: > > Hi everyone, > > > > First of all, thanks everyone that has participated in the sync so far. > > >

Re: Dropping JDK 8 support

+1 on dropping JDK 8 support with Iceberg 1.7 and I agree that Iceberg 2.0 should mostly focus around breaking API changes. On Wed, Jul 24, 2024 at 3:45 PM Jean-Baptiste Onofré wrote: > Hi Piotr > > I agree to drop JDK8 support (and related modules). > > You have my informal "+1" (this is not a

Re: [DISCUSS] Guidelines for committing PRs

Overall the proposal LGTM, thanks Micah On Fri, Jul 26, 2024 at 1:19 AM Micah Kornfield wrote: > As part of the bylaws discussions that have been happening, we are trying > to make small focused proposals to move things forward. As a first step > towards this I created a proposal for guidelines

Re: [VOTE] Drop Java 8 support in Iceberg 1.7.0

+1 (non-binding) for dropping JDK8 support with Iceberg 1.7.0 On Fri, Jul 26, 2024 at 1:29 PM Piotr Findeisen wrote: > Hi, > > Dropping support for building and running on Java 8 was discussed > previously on "Dropping JDK 8 support" and "Building with JDK 21" mail > threads. > > As JB kindly po

Re: [DISCUSS] Iceberg 1.6.1 release

I don't think we should be including general dependency updates in a patch release unless they are critical. On Mon, Jul 29, 2024 at 8:13 AM Jean-Baptiste Onofré wrote: > Hi, > > It would be great to include the Avro update in 1.6.1 release. > > I agree for a maintenance release on 1.6.x, but I

Re: [VOTE] Clarify "File System Tables" in the table spec

+1 (non-binding) On Thu, Aug 1, 2024 at 6:52 AM Micah Kornfield wrote: > +1 (non-binding) > > On Wed, Jul 31, 2024 at 5:12 PM Ryan Blue wrote: > >> As promised in the discussion thread, I've opened a PR to clarify the >> "File System Tables" section and mark it deprecated since there appears to

[DISCUSS] Changing namespace separator in REST spec

Hey everyone, The REST spec currently uses *%1F* as the UTF-8 encoded namespace separator for multi-part namespaces. This causes issues

Re: [DISCUSS] Changing namespace separator in REST spec

Here's the PR <https://github.com/apache/iceberg/pull/10837> that bumps Jetty and the Servlet API and reproduces #10338 <https://github.com/apache/iceberg/issues/10338> but it unfortunately requires to be built/executed with JDK17. On Thu, Aug 1, 2024 at 2:59 PM Eduard Tudenhöfn

Re: [DISCUSS] Changing namespace separator in REST spec

It's probably best to make this configurable rather than changing the separator in the spec. The server can communicate to the client which namespace separator should be used (via a config override), but still needs to support *%1F* as a fallback. I've opened #10877

Re: [DISCUSS] Changing namespace separator in REST spec

I've opened #10877 a few days ago to make the namespace separator configurable and let servers communicate to clients which separator should be used. Worth mentioning that this doesn't require any spec chance and it is backwards compatible with older c

Re: [DISCUSS] Changing namespace separator in REST spec

> Dmitri. > > [1] https://github.com/apache/iceberg/pull/10877 > > On Thu, Aug 8, 2024 at 6:28 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: > >> I've opened #10877 <https://github.com/apache/iceberg/pull/10877> a few >> days ago to mak

Re: [VOTE] Merge REST spec clarification on how servers should handle unknown updates/requirements

+1 On Tue, Aug 13, 2024 at 5:09 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > I've opened a PR [1] to clarify in the REST spec that if a server > receives an unknown update or requirement as part of any the commit > endpoints, that the server must fail with a 400 error response. > > Please vote

Re: Welcome Péter, Amogh and Eduard to the Apache Iceberg PMC

Thanks guys and also congrats to Péter and Amogh! On Wed, Aug 14, 2024 at 9:09 AM Fokko Driesprong wrote: > Congratulations and welcome! > > Kind regards, > Fokko > > Op wo 14 aug 2024 om 06:23 schreef Xuanwo : > >> Congrats! Thanks for your contribution. >> >> On Wed, Aug 14, 2024, at 11:32, Re

[DISCUSS] REST Endpoint discovery

Hey everyone, I'd like to propose a way for REST servers to communicate to clients what endpoints it supports via a new *endpoints* field in the *CatalogConfig* of the *v1/config* endpoint. This enables clients to make better decisions and clearly signal that a particular endpoint isn’t supported

Re: [DISCUSS] Describing REST Server capabilities

@Walaa: Those are all good feedback points that are appreciated. I realized that the capabilities probably add more complexity than necessary. As you mentioned, in the end we really want to know what endpoints a server supports. I've created a new design doc and opened a separate discussion thread

Re: [DISCUSS] Changing namespace separator in REST spec

e need for these changes entirely. >>> >>> An alternative to use an appropriate escaping mechanism, was turned down >>> on the same "Iceberg Catalog Sync" call. >>> >>> >>> Robert >>> >>> >>> [1] >>> https://g

Re: [DISCUSS] REST Endpoint discovery

On Wed, Aug 14, 2024 at 5:15 PM Dmitri Bourlatchkov wrote: > Hi Eduard, > > How is this proposal related to the Server Capabilities discussion? > > Thanks, > Dmitri. > > On Wed, Aug 14, 2024 at 5:14 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: >

Re: [DISCUSS] REST Endpoint discovery

uired in the Iceberg spec, and fix it to be consistent, assuming nobody > is taking a dependency on these operationIds right now. > > Personally speaking, I am pretty neutral on this topic, but curious what > everyone thinks. > > Best, > Jack Ye > > > > On Wed, Aug 14,

Re: [DISCUSS] Changing namespace separator in REST spec

gt; the door for inconsistency due to protocol level discrepancies. > > * Using servlet spec 6 with the ambiguous/suspiciuous validation breaks > all currently released clients. > > > There are catalogs that do allow this via Spark SQL: > CREATE NAMESPACE my_catalog.`my/ugly\nam

[VOTE] Make namespace separator configurable in REST Spec

Hey everyone, as I mentioned on the DISCUSS thread, this is providing a simple path forward for users of the V1 APIs (make the namespace separator *configurable* instead of *hardcoded*) that are either running into issue #10338 or will eventually wh

Re: [DISCUSS] REST Endpoint discovery

gt;>>> "GET": { >>>>> >>>>> "description": "List all namespaces" >>>>> >>>>> } >>>>> >>>>> }, >>>>> >>>>> {

Re: Iceberg-arrow vectorized read bug

Hey Steve, It's been a long time since I did some work on the iceberg-arrow module but I will try to find some time next week to analyze the problem in detail and see what options we have for fixing it. Thanks for your patience here. Eduard On Mon, Aug 12, 2024 at 9:00 PM Lessard, Steve wrote:

Re: [VOTE] Spec changes in preparation for v3

+1 On Tue, Aug 20, 2024 at 4:16 AM xianjin wrote: > +1 (non-binding) > Sent from my iPhone > > On Aug 20, 2024, at 7:56 AM, Manu Zhang wrote: > >  > +1 (non-binding) > > Micah Kornfield 于2024年8月20日 周二07:44写道： > >> +1 (non-binding) >> >> On Mon, Aug 19, 2024 at 4:33 PM Steve Zhang >> wrote: >>

Re: [DISCUSS] REST Endpoint discovery

m format). >>> >>> After reading the impl. PR [1] I think parsing is actually _not_ >>> necessary. The whole endpoint string is basically another form of an >>> endpoint "ID". >>> >>> If we agree that the values in the new endpoints config list s

Re: [DISCUSS] REST Endpoint discovery

add a new field parallel with the > endpoint array. Thanks a lot for driving this, Edward. > > Yufei > > > > On Tue, Aug 20, 2024 at 8:05 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: > >> @Karuppayya: I've answered your question in the doc. &g

[VOTE] REST Endpoint discovery

Hey everyone, I'd like to vote on PR #10928 which adds a way for REST servers to communicate to clients what endpoints it supports via a new *endpoints* field in the *CatalogConfig* of the *v1/config* endpoint. Discuss thread can be found here: https

Re: [VOTE] Release Apache Iceberg 1.6.1 RC1

@Piotr can you please elaborate which ORC update you are referring to? Or did you mean the Avro update (which I think we were planning for 1.6.2)? On Tue, Aug 20, 2024 at 7:05 PM Piotr Findeisen wrote: > Hi > > -1 (non-binding) > > I verified source tarball matches the git tag (except it > lacks

Re: [DISCUSS] Release source and binary verification

> > It’s not OK to include the gradle wrapper in the source release. A source > release can't include any jars with compiled code in them. Just to clarify that we're currently not adding the gradle wrapper in the source distribution. We have some custom code

Re: [VOTE] Release Apache Iceberg 1.6.1 RC1

I >> think we didn't cherry-pick it to 1.6.x branch. >> >> Kind regards, >> Fokko >> >> Op wo 21 aug 2024 om 09:34 schreef Eduard Tudenhöfner < >> etudenhoef...@apache.org>: >> >>> @Piotr can you please elaborate which ORC update you ar

Re: [EXTERNAL] Re: Iceberg-arrow vectorized read bug

ull > request to point out areas I know are problematic. > > > > -Steve Lessard, Teradata. > > > > > > > > > > *From: *Eduard Tudenhöfner > *Date: *Friday, August 16, 2024 at 2:45 AM > *To: *dev@iceberg.apache.org > *Subject: *[EXTERNAL] Re: Iceberg-a

Re: [VOTE] Release Apache Iceberg 1.6.1 RC1

Wed, 21 Aug 2024 at 09:45, Fokko Driesprong >> wrote: >> >> >> >> Hey Eduard, >> >> >> >> I think it relates to this PR. It contains a CVE and would be good to >> be backported. We wanted to include it in 1.6.1 if we needed another RC, >> b

Re: [VOTE] Release Apache Iceberg 1.6.1 RC2

+1 (binding) - verified signatures/checksums/license/tests On Thu, Aug 22, 2024 at 8:08 PM Carl Steinbach wrote: > Hi Everyone, > > I propose that we release the following RC as the official Apache Iceberg > 1.6.1 release. > > The commit ID is 8e9d59d299be42b0bca9461457cd1e95dbaad086 > * This

Re: [VOTE] REST Endpoint discovery

With 10 +1 votes (6 binding) this passes. I'll merge the PR. Thanks everyone On Wed, Aug 21, 2024 at 8:41 PM Anurag Mantripragada wrote: > +1 (non-binding) > > > Anurag Mantripragada > > > On Aug 21, 2024, at 3:11 AM, Sung Yun wrote: > > +1 (non-binding) > > Thank you Eduard! This is a great f

Re: [DISCUSS] Drop Hive 2 support

+1 on deprecating Hive 2 in Iceberg 1.7 and removing it in 1.8 On Mon, Sep 2, 2024 at 8:53 AM Jean-Baptiste Onofré wrote: > It sounds good. > > Thanks ! > Regards > JB > > On Mon, Sep 2, 2024 at 5:06 AM Manu Zhang wrote: > > > > Thanks JB. I think the last Hive 2 support will be Iceberg 1.7.x >

Re: [DISCUSS] REST: AuthManager API #10753

@Dmitri can you please share what overlap you see between these two PRs? The second PR aims at standardizing credentials used for different storage providers in the OpenAPI spec. On Thu, Sep 12, 2024 at 3:09 PM Dmitri Bourlatchkov wrote: > Apologies if I missed earlier discussions about this PR.

[DISCUSS] REST: Standardize vended credentials in Spec

Hey everyone, I'd like to propose standardizing the vended credentials used in loadTable / loadView responses. I opened #8 to track the proposal in GH. Please find the proposal doc here

Re: [DISCUSS] REST: Refreshing vended credentials

the >> cluster, and each compute node needs to fetch only the credentials >> independently. Such an API becomes handy to do improvements like caching. >> >> Best, >> Jack Ye >> >> [1] >> https://docs.aws.amazon.com/cli/latest/reference/lakeformation/get-tem

Re: [DISCUSS] REST: Standardize vended credentials in Spec

will not be to restrict what can be done, but to define a common > way of handling existing use cases. > > WDYT? > > Thanks, > Dmitri. > > On Thu, Oct 10, 2024 at 5:38 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: > >> Based on recent dis

[VOTE] Standardize vended credentials in OpenAPI spec

Hey everyone, I'd like to vote on #10722 , which has been open for quite a while now. I believe we're in agreement on how we want to standardize credentials in the OpenAPI spec. Please vote +1 if you generally agree with the path forward. Please vote

Re: [PROPOSAL] Partially Loading Metadata - LoadTable V2

Hey Haizhou, thanks for working on that proposal. I think my main concern with the current proposal is that it adds quite a lot of complexity at a bunch of places, since you'd need to partially update *TableMetadata*. Additionally, it requires a new endpoint. An alternative to that would be to do

Re: [VOTE] Table V3 Spec: Row Lineage

I left a few comments on the proposal but I'm overall +1 on the proposal On Thu, Oct 10, 2024 at 12:08 PM Jean-Baptiste Onofré wrote: > +1 > > I did a review on the proposal and it looks good to me. > > Regards > JB > > On Tue, Oct 8, 2024 at 3:55 PM Russell Spitzer > wrote: > > > > Hi Y'all! >

Re: [DISCUSS] Iceberg Summit 2025 ?

+1 for a hybrid event On Sun, Sep 29, 2024 at 4:51 AM Steven Wu wrote: > +1 for hybrid with in-person elements. > > On Sat, Sep 28, 2024 at 4:23 PM Matt Topol wrote: > >> +1 from me as well, I would love to attend an in person/hybrid iceberg >> summit. Workshops seem like a perfect way to help

Re: [VOTE] Standardize vended credentials in OpenAPI spec

With 8 +1 votes the VOTE passed. Thanks everyone On Fri, Oct 18, 2024 at 12:34 AM Jack Ye wrote: > +1 (binding) > > Best, > Jack Ye > > On Thu, Oct 17, 2024 at 3:05 PM Dmitri Bourlatchkov > wrote: > >> +1 (non-binding) >> >> Cheers, >> Dmit

[VOTE] Endpoint for refreshing vended credentials

Hey everyone, I'd like to vote on #11281 , which introduces a new endpoint and allows retrieving/refreshing vended credentials for a given table. Please vote +1 if you generally agree with the path forward. Please vote in the next 72 hours [ ] +1, c

Re: [DISCUSS] REST: Standardize vended credentials in Spec

the config more human-readable without adding too much >>> processing burden. I hope the standard is well supported by most language >>> libraries now. It is certainly supported by java. WDYT? >>> >>> Thanks, >>> Dmitri. >>> >>> On Fri, Sep

Re: [Discuss] Replace Hadoop Catalog Examples with JDBC Catalog in Documentation

I would prefer to advocate for the REST catalog in those examples/docs (similar to how the Spark quickstart example uses the REST catalog). The docs could then refer to the quickstart example to indicate what's required in terms of services to be start

[DISCUSS] REST: Refreshing vended credentials

Hey everyone, I'd like to propose a mechanism and changes in order to be able to refresh vended credentials for tables. Please find the proposal doc here . The proposal requires a spec change, which

Re: [VOTE] Endpoint for refreshing vended credentials

refreshed, when any of the > previous credentials expires. I think this is suboptimal (but can probably > be made to work in most practical cases). > > Cheers, > Dmitri. > > On Mon, Oct 21, 2024 at 6:07 AM Eduard Tudenhöfner < > etudenhoef...@apache.org> wrote: > >>

Re: [VOTE] Endpoint for refreshing vended credentials

i Bourlatchkov >> wrote: >> >> Thanks for the reply Eduard! >> >> I think it is fine to defer fine-tuning credential refreshes to a later >> PR. >> >> I'm upgrading my vote to +1 (non-binding). >> >> Cheers, >> Dmitri. >>

Re: [PROPOSAL] Refactore use of Guava Lists.*

Hey JB, I don't think we're ever using e.g. *Lists.newArrayList()* without the diamond syntax in the codebase, so it's typically always *List list = Lists.newArrayList()*. So I wonder how much of an issue that actually is? Do you have examples in the codebase that don't use the diamond syntax and

Re: [VOTE] Deletion Vectors in V3

+1 On Wed, Oct 30, 2024 at 7:28 AM Jean-Baptiste Onofré wrote: > +1 (non binding) > > Regards > JB > > On Tue, Oct 29, 2024 at 10:45 PM Anton Okolnychyi > wrote: > > > > Hi folks, > > > > We have been discussing the new layout for position deletes in V3 for a > while now. It seems the community

Re: [VOTE] Release Apache Iceberg 1.7.0 RC0

+1 (binding) Verified signature/checksum/license and build/test with JDK17 On Mon, Nov 4, 2024 at 1:15 AM Honah J. wrote: > +1 (binding) > > Verified signature/checksum/license/build/test with JDK17 > > Best regards, > Honah > > On Sun, Nov 3, 2024 at 10:47 AM Cheng Pan wrote: > >> FYI, I jus

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

@Matt yes the licensing is a blocker unfortunately, so we should cancel the RC and start an RC1 vote once this is fixed. I should have properly voted in the earlier email, but will do it now. -1 (binding) due to the licensing On Tue, Nov 12, 2024 at 5:47 PM Jean-Baptiste Onofré wrote: > -1 (non

Re: [DISCUSS] Spark 3.3 support?

+1 to deprecating and removing it On Wed, Nov 13, 2024 at 5:03 PM Anton Okolnychyi wrote: > What do folks think about our Spark 3.3 support? Spark 3.3.0 was released > in June, 2022. Given the 18 month maintenance period in Spark, it is no > longer maintained. The last release was in December, 2

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

I also ran into the same issues that Kevin pointed out earlier while verifying the release, but apart from that all tests passed. I think the LICENSE file needs to be updated as it contains Java-specific things, which can be seen in https://github.com/apache/iceberg-go/blob/main/LICENSE#L216-L315.

Re: [DISCUSS] REST: Standardize vended credentials in Spec

gt; hopefully make the config more human-readable without adding too much > processing burden. I hope the standard is well supported by most language > libraries now. It is certainly supported by java. WDYT? > > Thanks, > Dmitri. > > On Fri, Sep 13, 2024 at 12:13 PM Eduard Tu

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

+1 (binding) On Tue, Oct 1, 2024 at 1:45 AM Micah Kornfield wrote: > I'm -0.0 as worded currently. I think there are some more aspects that > should be defined for date->timestamp/timestamp_ns promotion (left comments > on the PR). The addition of an Unknown type seems like a good addition. >

Re: Code structuring question

Having a separate "api" gradle module seems a lot of work and so starting with an "api" package seems to make sense to me. This obviously has the side-effect you mentioned (requiring package-private classes to be public). I don't think we have anything in particular in the Iceberg codebase that wo

Re: REST Catalog based Integration Test for Query Engines

Thanks for looking into this Haizhou. I'll take a closer look at the PRs this/next week. Eduard On Thu, Sep 19, 2024 at 2:22 AM Haizhou Zhao wrote: > Hello dev-list, > > *What* > I'm looking for issues and PRs reviews from the community to enable REST > Catalog based Integration Test for Query

Re: [DISCUSS] Remove iceberg-pig module ?

+1 for marking the project deprecated (in 1.7.0) and dropping it in the next release (1.8.0) On Thu, Oct 17, 2024 at 4:36 PM Russell Spitzer wrote: > +1 (oink) > > If anyone really cares please chime in but seriously we should drop it > > On Thu, Oct 17, 2024 at 8:07 AM Jean-Baptiste Onofré > w

Re: [VOTE] Release Apache Iceberg 1.7.0 RC1

+1 (binding) Verified signature/checksum/license and build/test with JDK17 On Thu, Nov 7, 2024 at 3:11 AM Daniel Weeks wrote: > +1 (binding) > > Verified sigs/sums/license/build/test (Java 17) > > -Dan > > On Wed, Nov 6, 2024 at 3:23 PM Jack Ye wrote: > >> +1 (binding) >> >> - Verified signatu

Re: Duplicates are getting inserted into Iceberg tables even after de-duplication

I do recall an issue where duplicate data/delete files where possible, but I'm not sure if that's the underlying cause in your case. The issue was fixed by #10007 and was shipped with Iceberg 1.6.0. On Thu, Nov 7, 2024 at 11:12 PM Lewis, William wrot

Re: [DISCUSS] Apache Iceberg Summit 2025 - Selection Committee

Thanks for organizing this and I'd like to volunteer to help out where I can. On Wed, Nov 27, 2024 at 9:16 AM Christian Thiel wrote: > Hey JB, > > happy to help any way I can. Thanks for organizing this! > > Best, > Christian > > On 27. Nov 2024, at 07:52, Fokko Driesprong wrote: > > Hey JB, >

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC2

+1 (binding) verified checksum, signatures and tests Thanks Matt for doing the release On Thu, Nov 14, 2024 at 9:24 PM Kevin Liu wrote: > +1 (non-binding) > > Verified checksum, signature, tests > > I modified the verify_rc script to use artifacts from the apache dist > https://github.com/apac

Re: [DISCUSS] Iceberg 1.7.1 release

@Aihua I don't think #11324 is a good candidate to include for 1.7.1 as a patch release typically should only include bug fixes and not new features. We are planning to do a 1.8.0 in the next few weeks and so #11324 could be shipped with that. Thanks, Eduard On Fri, Nov 15, 2024 at 6:23 AM Aihua

Re: [PROPOSAL] Store Iceberg build scans on ASF Develocity (ge.apache.org)

I think that's a good idea, so +1 from my side. On Mon, Nov 25, 2024 at 11:10 AM Jean-Baptiste Onofré wrote: > Hi folks, > > The ASF is hosting a Gradle Develocity instance where we can store our > build scans. > > It's hosted on https://ge.apache.org. > Several projects are using it (Apache Bea

Re: [VOTE] Release Apache Iceberg 1.7.1 RC1

I actually verified everything (checksums, license checks, tests) last week but forgot to vote, sorry about that. Also thanks for running the release Bryan. +1 (binding) On Tue, Dec 3, 2024 at 8:37 AM Fokko Driesprong wrote: > Hey Bryan, > > Thanks for running the release! +1 binding from my en

Re: [VOTE] Update partition stats spec for V3

+1 On Mon, Feb 3, 2025 at 8:33 PM Dongjoon Hyun wrote: > +1 for the proposal. > > Dongjoon > > > On Mon, Feb 3, 2025 at 2:35 PM ConradJam wrote: > > > > > +1 (non-binding) > > > > > > Steven Wu 于2025年2月3日周一 14:08写道： > > > > > >> +1 > > >> > > >> The spec change makes sense. left a question in

Re: [VOTE] Add Geometry and Geography types for V3

+1 On Sat, Feb 8, 2025 at 1:02 PM Fokko Driesprong wrote: > +1 > > Op za 8 feb 2025 om 08:08 schreef Péter Váry >: > >> +1 >> >> On Fri, Feb 7, 2025, 21:20 Kevin Liu wrote: >> >>> +1 (non-binding) >>> It's great to see support for more data types in both parquet and >>> Iceberg! >>> >>> Best,

Re: [VOTE] Simplify multi-arg table metadata

+1 On Mon, Feb 10, 2025 at 7:40 AM Péter Váry wrote: > +1 > > On Mon, Feb 10, 2025, 03:44 Manu Zhang wrote: > >> +1 (non-binding) >> >> On Mon, Feb 10, 2025 at 10:25 AM roryqi wrote: >> >>> +1 >>> >>> xianjin 于2025年2月10日周一 10:02写道： >>> +1 (non-binding) On Mon, Feb 10, 2025 at 2

Re: Welcome Huaxin Gao as a committer!

Congratulations Huaxin On Thu, Feb 6, 2025 at 2:40 PM Rodrigo Meneses wrote: > Congrats and best wishes !!! > > On Thu, Feb 6, 2025 at 5:04 AM Gidon Gershinsky wrote: > >> Congrats Huaxin! >> >> Cheers, Gidon >> >> >> On Thu, Feb 6, 2025 at 2:46 PM Tushar Choudhary < >> tushar.choudhary...@gmai

Re: [VOTE] Add RemoveSchemas update type to REST spec

+1 On Tue, Feb 11, 2025 at 10:40 AM Gabor Kaszab wrote: > Hi Iceberg Community, > > I'm working on removing the unused schemas from the table metadata when > running snapshot expiration. One part of this work is a change in REST spec > to add a new update type for removing schemas. > > I'd like

Re: New committer: Scott Donnelly

Congrats Scott! On Wed, Dec 11, 2024 at 7:35 AM roryqi wrote: > Congrats! > > Fenil Jain 于2024年12月11日周三 14:26写道： > >> Congratulations Scott! >> >> On Wed, Dec 11, 2024 at 8:56 AM Renjie Liu >> wrote: >> > >> > Hey everyone, >> > >> > The Project Management Committee (PMC) for Apache Iceberg ha

Re: New committer: Matt Topol

Congrats Matt! On Wed, Dec 11, 2024 at 6:41 AM Honah J. wrote: > Congratulations, Matt! > > On Tue, Dec 10, 2024 at 7:51 PM Fenil Jain wrote: > >> Congratulations Matt! >> >> On Tue, Dec 10, 2024 at 3:56 PM Fokko Driesprong >> wrote: >> > >> > Hey everyone, >> > >> > The Project Management Com

Re: [ANNOUNCE] Apache Iceberg release 1.7.1

Thanks Bryan and everyone else for making this release happen. On Tue, Dec 10, 2024 at 12:27 AM Yuya Ebihara wrote: > Thank you Brian! Trino project had waited for 1.7.1 that fixes namespace > regression. >

Re: 1.7.1 breaking change related to ADLS support

I agree with Fokko in that we should do a 1.7.2 release for people using ADLSFileIO On Tue, Dec 17, 2024 at 9:58 AM Fokko Driesprong wrote: > Thanks for raising this Alex, > > I suggest doing a 1.7.2 patch release since we don't want to leave the > 1.7.x version in a broken state for the ADLSFil

  1   2   >