Re: [DISCUSS] Gradle and Maven runner plugins

2025-07-17 Thread Alexandre Dutra
Hi all, If the runner plugins can save us from the dependency nightmare between Spark and Quarkus, imo that's a huge selling point. An example: the QuarkusSparkIT test lives in its own module (!), precisely because of dependency issues. Would the runner plugin allow us to move that test back to t

Re: [DISCUSS] Gradle and Maven runner plugins

2025-07-17 Thread Robert Stupp
> Would the runner plugin allow us to move that test back to the > polaris-runtime-service module? Yes On Thu, Jul 17, 2025 at 12:52 PM Alexandre Dutra wrote: > > Hi all, > > If the runner plugins can save us from the dependency nightmare > between Spark and Quarkus, imo that's a huge selling p

Re: [DISCUSS] Gradle and Maven runner plugins

2025-07-17 Thread Dmitri Bourlatchkov
Good point on QuarkusSparkIT! Cheers, Dmitri. On Thu, Jul 17, 2025 at 6:52 AM Alexandre Dutra wrote: > Hi all, > > If the runner plugins can save us from the dependency nightmare > between Spark and Quarkus, imo that's a huge selling point. > > An example: the QuarkusSparkIT test lives in its o

Secrets management

2025-07-17 Thread Robert Stupp
Hi all, We now have an interface `UserSecretsManager` for safe storage of secrets (thanks Dennis!). With that interface we just have one implementation named `UnsafeInMemorySecretsManager`, which is pretty much what the name says. For posterity: it does not persist secrets but only keeps those in

Re: [DISCUSS] Gradle and Maven runner plugins

2025-07-17 Thread Robert Stupp
That's correct. That "dependency hell" wouldn't exist. Another aspect is that the integration tests all require their "custom" Quarkus build, which would not be necessary with the apprunner, leading to faster CI. On Wed, Jul 16, 2025 at 3:42 PM Dmitri Bourlatchkov wrote: > > I was reviewing our

Re: Discussion Regarding Events Instrumentation - GH PR #1904

2025-07-17 Thread Robert Stupp
Events, once they're "in the code base" are pretty much a "public facing API", which I think justifies careful handling and re-iteration, especially given the IRC work. With respect to the implementation, I have a few concerns regarding security and behavioral consistency. Secrets are published v

Re: Discussion Regarding Events Instrumentation - GH PR #1904

2025-07-17 Thread Robert Stupp
Actually, thinking a bit more about this, requiring that event payloads have strict and direct dependencies on Iceberg or Polaris types can be quite a disadvantage mid/long-term. Having to adjust all event listener implementations just because something in some unrelated type changed feels odd. Whi

Re: [Community Meeting] NoSQL Support in Apache Polaris

2025-07-17 Thread Jean-Baptiste Onofré
Hi Thanks everyone who joined the NoSQL Community meeting yesterday. Here's the record: https://drive.google.com/file/d/1XN1ng8ORB3-jCAeWmCt0vOn3zyZmvRWh/view?usp=sharing In terms of next steps: 1. Adam will start a thread on the dev mailing list to gather feedback and share the main updates on

[DISCUSS] Apache Polaris (incubating) 1.1.0 release mid August ?

2025-07-17 Thread Jean-Baptiste Onofré
Hi folks, I propose to have 1.1.0-incubating release around August 20. On github: 1. I updated 1.1.0 milestone due date 2. I will move the open issues still on 1.0.0 milestone to 1.1.0 3. I will close the 1.0.0 milestone (as it has been released) My main focus is to review/update/improve the rel

Re: Discussion Regarding Events Instrumentation - GH PR #1904

2025-07-17 Thread Adnan Hemani
Hi Robert, Thanks for sharing your suggestions on this topic! I’ll respond in-line: >> Secrets are published via events (e.g. from "rotateCredentials()"). >> That should IMHO never happen. I see your comment on the PR about this - this is my oversight, it should only be returning Principal. Ap

[DISCUSS] Quick sketch of next-steps to implement table/namespace-level RBAC and credential vending for Polaris Federation

2025-07-17 Thread Dennis Huo
Hey all, I sketched out some design details for next-step features in Polaris Federation based on discussion with Pooja, Rulin, Harish, and Teja: https://docs.google.com/document/d/19Vm0Uy-EyEYtgd2-bZpPICODOB3rYQJvHeqL1ODOOLc/edit?tab=t.0 At a high level, currently the REST-based Federation MVP b

Re: [DISCUSS] Hive Catalog federation in Polaris

2025-07-17 Thread Dennis Huo
A lot of the concerns seem to arise from the fact that Polaris essentially has two distinct deployment modes: 1. Provide a highly isolated multi-tenant (multi-realm) Iceberg REST service with a tightly curated set of secure integrations to external dependencies 2. Provide a turnkey Iceberg REST se

Re: [DISCUSS] Polaris Delegation Service for Long-Running Tasks

2025-07-17 Thread William Hyun
Hello all, I've put together a proof-of-concept to help us explore some of the implementation details and move the conversation forward. This PoC demonstrates a basic task delegation flow from a Polaris instance to a Delegation Service instance. You can get started with the quickstart guide and r