Hi all, Thanks for the excellent discussion so far! I added this topic to the agenda for today's meeting, in order to get some alignment before diving into a design doc or formal proposal. I hope that's OK for everybody.
Thanks, Alex On Sun, Nov 9, 2025 at 6:54 AM Yufei Gu <[email protected]> wrote: > > Thanks for the great discussion. +1 on clarifying the boundary and mutually > exclusive modes, so that there is no confusion and chaos. > > Can we have a short design doc to facilitate further discussion? > Particularly. it'd be nice to clarify authN and authZ modes Polaris > supports like the following table: > > Mode Principal/Role persistence Privilege enforcement Who decides? > *Native* Persisted Polaris RBAC Polaris > *Federated* Persisted/synced Polaris RBAC Polaris > *Detached* Ephemeral External PDP (e.g. OPA, Ranger) External > > Once we clarified different modes, we could start to figure out the > implications and correspondent changes. Things we might consider: > 1. Do we allow mixed mode, e.g., some operations are with detached mode, > others are with native mode? I guess this is pretty complex, we may not do > that if there are no strong use cases. > 2. How do we handle the audit in case of detached mode? Can we still get > the consistent principal names? > > I'm happy to jump on a dedicated call for this. > > Yufei > > > On Sat, Nov 8, 2025 at 7:20 PM Sung Yun <[email protected]> wrote: > > > Thank you for suggesting this Alex, this is a great discussion. > > > > I see the value of persisted federated roles when Polaris is the primary > > place where catalog privileges are managed and external IdPs just supply > > identity and groups. In that model, stable role entities in Polaris are > > useful and provide a clear governance boundary. > > > > For the OPA-backed PolarisAuthorizer use case, though, we are moving > > toward a setup where: > > - Principals are not persisted in Polaris. > > - Role and attribute evaluation happen in an external Policy Decision > > Point such as OPA. > > - Polaris simply enforces authorization decisions on its REST resources. > > > > In this pattern, persisted federated roles become less compelling and can > > even overlap with external policy configuration. So this “detached > > principal” concept resonates with our use case… it makes the externalized > > responsibility model explicit instead of overloading federated roles to > > serve both the Polaris-managed RBAC and externally-managed Policy Decision > > Point use cases. > > > > So IMHO, it might be worth clarifying the model boundaries so both modes > > are first-class: > > - “Federated” for external identity with locally managed privileges in > > Polaris. > > - “Detached” for external identity with externally managed authorization > > and no local persistence. > > > > That distinction could help keep the semantics clean while still > > minimizing conceptual sprawl. > > > > If it would be helpful, would it make sense to organize a call to discuss > > these concepts for those that are interested? (The usual Polaris Sync time > > slot doesn’t work for me unfortunately, and I’d love to make a case for > > this principal type to be supported) > > > > Cheers, > > Sung > > > > On 2025/11/08 01:09:34 Michael Collado wrote: > > > I see the distinction you're raising. It's a good point, though I'm > > > hesitant to introduce a third principal type to the code base. The most > > > notable thing about federated principals and roles is that role grants > > are > > > never managed through the Polaris API. This would be true even for > > > "detached" principals/roles. Federated principals (in the current code > > > base) also can't be persisted (except possibly through some alternative > > > process, such as SCIM). The main difference is that federated roles are > > > expected to be persisted, but "detached" roles are not. Can we unify > > these > > > concepts so that we can support both ephemeral roles and persisted roles > > > without introducing a third type of role? > > > > > > Mike > > > > > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <[email protected]> > > wrote: > > > > > > > Hi all, > > > > > > > > > The code change in #1353 prohibited federated principals from being > > > > persisted at all. > > > > > > > > Indeed! But I think it's important to distinguish between "federated" > > > > and "detached". Here's how I see the difference: > > > > > > > > 1) Federated Principals and Roles > > > > > > > > This approach involves pre-synchronizing principals and roles with an > > > > external system before they can access Polaris. This concept was > > > > outlined in an older design document [1] but never fully implemented. > > > > Typically, this would be achieved using SCIM (eventually) or a > > > > periodic synchronization script run by an administrator. > > > > > > > > Re: federated principals persistence. PR 1353 [2] assumes > > > > just-in-time (JIT) creation of federated principals, meaning the > > > > principal would be materialized only upon first access. That's not the > > > > case today though. I personally wonder if it would be simpler to also > > > > allow for principal synchronization via the management API. (I'm happy > > > > to resume this conversation later, but that's not the topic here.) > > > > > > > > 2) Detached Principals and Roles > > > > > > > > This scenario implies that no identity management information > > > > whatsoever is stored in Polaris. This fits use cases where the > > > > authorizer does not rely on Polaris's default privilege model: the OPA > > > > authorizer is one example of that. This is the specific use case > > > > currently under discussion. > > > > > > > > Both flavors of principals are important for Polaris, and I would > > > > gladly resume the work on federated ones. But, it seems to me that > > > > enabling detached principals is even more urgent today. WDYT? > > > > > > > > > I'd like this lookup to be covered by a feature flag [...] it would > > be > > > > nice to allow skipping the unnecessary database hit. WDYT? > > > > > > > > Yes, that is a very good suggestion! > > > > > > > > Thanks, > > > > Alex > > > > > > > > [1]: > > > > > > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5 > > > > [2]: https://github.com/apache/polaris/pull/1353 > > > > > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado < > > [email protected]> > > > > wrote: > > > > > > > > > > The code change in #1353 prohibited federated principals from being > > > > > persisted at all. Federated Principal Roles are persisted, but at > > this > > > > time > > > > > (AFAIK), we can't persist federated principals. > > > > > > > > > > Mike > > > > > > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <[email protected] > > > > > > > wrote: > > > > > > > > > > > Hi Alex, > > > > > > > > > > > > This proposal sounds very reasonable to me. > > > > > > > > > > > > I believe we had some recent user interest (in slack) for running > > > > Polaris > > > > > > with Keycloak. From my POV, implementing your proposal should > > > > > > significantly simplify that user's deployment. > > > > > > > > > > > > As for item (3b), I'd like this lookup to be covered by a feature > > flag. > > > > > > Admin users who do not import / sync Principals know that this > > lookup > > > > will > > > > > > always fail, so it would be nice to allow skipping the unnecessary > > > > database > > > > > > hit. WDYT? > > > > > > > > > > > > Thanks, > > > > > > Dmitri. > > > > > > > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <[email protected] > > > > > > > wrote: > > > > > > > > > > > > > Hello everyone, > > > > > > > > > > > > > > I'd like to propose an enhancement to Polaris's authentication > > > > > > > process: the introduction of "detached" (non-persisted) > > principals. > > > > > > > > > > > > > > A detached principal is different from a federated principal. The > > > > > > > former isn't persisted at all, whereas the latter is persisted > > (with > > > > > > > federated flag = true), and possibly synchronized with some > > external > > > > > > > system (e.g. via SCIM). > > > > > > > > > > > > > > Motivation: > > > > > > > > > > > > > > Currently, Polaris requires principals to be pre-existing for > > > > > > > successful authentication – even those marked as "federated". The > > > > > > > default authenticator always performs a lookup by ID or name, and > > > > > > > fails if the principal isn't found. > > > > > > > > > > > > > > This presents a challenge for users who utilize external > > > > > > > authentication and authorization systems, such as Keycloak + OPA, > > > > > > > where Polaris acts as a passthrough between authn and authz. > > > > > > > > > > > > > > In such scenarios, principals still need to be created in advance > > > > > > > within Polaris, which doesn't make much sense. > > > > > > > > > > > > > > Proposal: > > > > > > > > > > > > > > 1) Introduce a new property: PolarisCredential.isExternal(). > > > > > > > > > > > > > > 2) During authentication, set isExternal=false on the > > > > > > > PolarisCredential if Polaris is the IDP (internal auth); > > otherwise, > > > > > > > set isExternal=true. > > > > > > > > > > > > > > 3) In DefaultAuthenticator: > > > > > > > > > > > > > > a) If PolarisCredential.isExternal() is false, continue with > > the > > > > > > > existing logic (lookups by ID and name). > > > > > > > > > > > > > > b) If PolarisCredential.isExternal() is true, perform the > > usual > > > > > > > lookups. If they succeed, continue with the existing logic. If > > they > > > > > > > fail though, don't fail the authentication. Instead, create the > > > > > > > principal and assign roles directly from the token. > > > > > > > > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid > > re-resolving > > > > > > > the principal and its roles (as this would fail for detached > > > > > > > principals). > > > > > > > > > > > > > > I believe this change will not impact internal authentication > > with > > > > the > > > > > > > default authorizer. It also won't impact people using external > > IDPs > > > > > > > with pre-synchronized federated principals: the default > > authorizer > > > > > > > would work just fine. > > > > > > > > > > > > > > However, for external IDPs combined with custom authorizers, this > > > > > > > change would significantly improve the user experience by > > eliminating > > > > > > > the need to persist the principal and its roles within Polaris. > > > > > > > > > > > > > > What do you all think? > > > > > > > > > > > > > > Thanks, > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > >
