Re: [PROPOSAL] Commit Deconfliction

Russell, can you clarify? The proposal as it stands would make one of the conflicting commit's resulting snapshot the latest snapshot, superseding data added in the conflicting commits, leading to data loss ... that's correct, isn't it? On Tue, Jul 8, 2025 at 11:22 AM Russell Spitzer wrote: > > I

Re: [PROPOSAL] Commit Deconfliction

Hi Eric, Just to clarify: Are you proposing Polaris Server to reconcile metadata changes from two conflicting commit requests or are you proposing the clients to do that? In other words, who will make the final metadata JSON? Clients (and the server merely commits it) or the Polaris Server itself

[DISCUSS] Adding pathStyleAccess to AwsStorageConfigInfo

Hi All, I propose to add a boolean `pathStyleAccess` optional parameter to AwsStorageConfigInfo in PR [2012]. The main idea is to support non-AWS S3 implementations for [1530] but the new parameter is meaningful in AWS SDK S3 clients in general. If set, the value will be propagated to Iceberg RE

Re: [PROPOSAL] Commit Deconfliction

The general feature of allowing clients to declare that they take responsibility for overwriting each other's change is fine from my POV. As previous messages point out there are use cases for it. However, the term "deconfliction" does not sound perfect to me in this case. From my POV deconflictio

Re: Remove k8s reference for kind

Hi, I also have a preference for minikube – even if Kind has the ability of launching multi-node clusters, which minikube can't do. Historically, Kind was used in one single place: the run.sh script at the root of the repo. I think it was chosen so that we can launch a local Docker registry (cf.

Re: [PROPOSAL] Commit Deconfliction

I do like this proposal because it essentially avoids all the issues that Robert mentions by instead just offering the ability for a client to decide in advance which commits would succeed. Leaving more advanced automatic or server side determined deconfliction is a good future direction but I thin

Re: [PROPOSAL] Commit Deconfliction

The general idea to resolve commit-conflicts in Polaris is fine. However I miss some information about the tricky details. The tricky part is how to detect and in turn how to resolve those conflicts. That requires knowledge of the change being performed and its context. While it sounds simple to

Re: [PROPOSAL] Commit Deconfliction

Hi all, What Russel wrote in (2) is correct -- this will essentially be a best-effort deconfliction, but my hope is that we can expand the supported cases over time. It may be helpful to make a distinction between commits which are "eligible for deconfliction" from commits that "the server knows h

Proposal: Optional clientId / clientSecret in createPrincipal API for Migration Support

Hi all, Following up on the suggestion from the discussion here — thank you for the feedback so far. We’re currently migrating our self-hosted Polaris service from version 0.9 (EclipseLink-based metastore) to version 1.0 (JDB

Re: [PROPOSAL] Commit Deconfliction

For me there are two big use cases for this: 1. Simple overwrite. I may have several jobs, for example one that does TTL , the command it runs is idempotent and always deletes all rows / files before a certain point. The other is a merge/update command. In this situation I don't even need to recon

Re: [DISCUSS] Hive Catalog federation in Polaris

HMS integration is a key step toward one of Polaris’s critical missions: helping users move off HMS. It brings clear value by aligning with our long-term direction. I’m not too concerned about hive.xml, most of its configurations can be dynamically injected at runtime. The real challenge lies in K

Re: Proposal: Optional clientId / clientSecret in createPrincipal API for Migration Support

Hi Arun, Thank you for starting this discussion! I did some poking about Keycloak and it looks like Keycloak allows user-provided Client IDs. I think it should be fine for Polaris to accept user-provided Client IDs in the Principal management API. I suppose we may want to impose some restriction

Re: [DISCUSS] Prepare for 1.0 Release

Hi everyone, Please take 5 minutes to check the 1.0.0 release notes: https://docs.google.com/document/d/1JDVdQraoEhOIv7agy7WzIuBQdW0_16jW-DBrnanuW7A/edit?tab=t.0. We will publish it right after 1.0.0 was done. Yufei On Thu, Jun 26, 2025 at 10:31 AM Yufei Gu wrote: > Thanks for the validation,