Thank you for suggesting this Alex, this is a great discussion. I see the value of persisted federated roles when Polaris is the primary place where catalog privileges are managed and external IdPs just supply identity and groups. In that model, stable role entities in Polaris are useful and provide a clear governance boundary.
For the OPA-backed PolarisAuthorizer use case, though, we are moving toward a setup where: - Principals are not persisted in Polaris. - Role and attribute evaluation happen in an external Policy Decision Point such as OPA. - Polaris simply enforces authorization decisions on its REST resources. In this pattern, persisted federated roles become less compelling and can even overlap with external policy configuration. So this “detached principal” concept resonates with our use case… it makes the externalized responsibility model explicit instead of overloading federated roles to serve both the Polaris-managed RBAC and externally-managed Policy Decision Point use cases. So IMHO, it might be worth clarifying the model boundaries so both modes are first-class: - “Federated” for external identity with locally managed privileges in Polaris. - “Detached” for external identity with externally managed authorization and no local persistence. That distinction could help keep the semantics clean while still minimizing conceptual sprawl. If it would be helpful, would it make sense to organize a call to discuss these concepts for those that are interested? (The usual Polaris Sync time slot doesn’t work for me unfortunately, and I’d love to make a case for this principal type to be supported) Cheers, Sung On 2025/11/08 01:09:34 Michael Collado wrote: > I see the distinction you're raising. It's a good point, though I'm > hesitant to introduce a third principal type to the code base. The most > notable thing about federated principals and roles is that role grants are > never managed through the Polaris API. This would be true even for > "detached" principals/roles. Federated principals (in the current code > base) also can't be persisted (except possibly through some alternative > process, such as SCIM). The main difference is that federated roles are > expected to be persisted, but "detached" roles are not. Can we unify these > concepts so that we can support both ephemeral roles and persisted roles > without introducing a third type of role? > > Mike > > On Wed, Nov 5, 2025 at 8:35 AM Alexandre Dutra <[email protected]> wrote: > > > Hi all, > > > > > The code change in #1353 prohibited federated principals from being > > persisted at all. > > > > Indeed! But I think it's important to distinguish between "federated" > > and "detached". Here's how I see the difference: > > > > 1) Federated Principals and Roles > > > > This approach involves pre-synchronizing principals and roles with an > > external system before they can access Polaris. This concept was > > outlined in an older design document [1] but never fully implemented. > > Typically, this would be achieved using SCIM (eventually) or a > > periodic synchronization script run by an administrator. > > > > Re: federated principals persistence. PR 1353 [2] assumes > > just-in-time (JIT) creation of federated principals, meaning the > > principal would be materialized only upon first access. That's not the > > case today though. I personally wonder if it would be simpler to also > > allow for principal synchronization via the management API. (I'm happy > > to resume this conversation later, but that's not the topic here.) > > > > 2) Detached Principals and Roles > > > > This scenario implies that no identity management information > > whatsoever is stored in Polaris. This fits use cases where the > > authorizer does not rely on Polaris's default privilege model: the OPA > > authorizer is one example of that. This is the specific use case > > currently under discussion. > > > > Both flavors of principals are important for Polaris, and I would > > gladly resume the work on federated ones. But, it seems to me that > > enabling detached principals is even more urgent today. WDYT? > > > > > I'd like this lookup to be covered by a feature flag [...] it would be > > nice to allow skipping the unnecessary database hit. WDYT? > > > > Yes, that is a very good suggestion! > > > > Thanks, > > Alex > > > > [1]: > > https://docs.google.com/document/d/15_3ZiRB6Lhzw0nxij341QUdxEIyFGTrI9_18bFIyJVo/edit?tab=t.0#heading=h.cu1a1acu4lc5 > > [2]: https://github.com/apache/polaris/pull/1353 > > > > On Tue, Nov 4, 2025 at 10:05 PM Michael Collado <[email protected]> > > wrote: > > > > > > The code change in #1353 prohibited federated principals from being > > > persisted at all. Federated Principal Roles are persisted, but at this > > time > > > (AFAIK), we can't persist federated principals. > > > > > > Mike > > > > > > On Tue, Nov 4, 2025 at 9:28 AM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > > > Hi Alex, > > > > > > > > This proposal sounds very reasonable to me. > > > > > > > > I believe we had some recent user interest (in slack) for running > > Polaris > > > > with Keycloak. From my POV, implementing your proposal should > > > > significantly simplify that user's deployment. > > > > > > > > As for item (3b), I'd like this lookup to be covered by a feature flag. > > > > Admin users who do not import / sync Principals know that this lookup > > will > > > > always fail, so it would be nice to allow skipping the unnecessary > > database > > > > hit. WDYT? > > > > > > > > Thanks, > > > > Dmitri. > > > > > > > > On Tue, Nov 4, 2025 at 11:53 AM Alexandre Dutra <[email protected]> > > wrote: > > > > > > > > > Hello everyone, > > > > > > > > > > I'd like to propose an enhancement to Polaris's authentication > > > > > process: the introduction of "detached" (non-persisted) principals. > > > > > > > > > > A detached principal is different from a federated principal. The > > > > > former isn't persisted at all, whereas the latter is persisted (with > > > > > federated flag = true), and possibly synchronized with some external > > > > > system (e.g. via SCIM). > > > > > > > > > > Motivation: > > > > > > > > > > Currently, Polaris requires principals to be pre-existing for > > > > > successful authentication – even those marked as "federated". The > > > > > default authenticator always performs a lookup by ID or name, and > > > > > fails if the principal isn't found. > > > > > > > > > > This presents a challenge for users who utilize external > > > > > authentication and authorization systems, such as Keycloak + OPA, > > > > > where Polaris acts as a passthrough between authn and authz. > > > > > > > > > > In such scenarios, principals still need to be created in advance > > > > > within Polaris, which doesn't make much sense. > > > > > > > > > > Proposal: > > > > > > > > > > 1) Introduce a new property: PolarisCredential.isExternal(). > > > > > > > > > > 2) During authentication, set isExternal=false on the > > > > > PolarisCredential if Polaris is the IDP (internal auth); otherwise, > > > > > set isExternal=true. > > > > > > > > > > 3) In DefaultAuthenticator: > > > > > > > > > > a) If PolarisCredential.isExternal() is false, continue with the > > > > > existing logic (lookups by ID and name). > > > > > > > > > > b) If PolarisCredential.isExternal() is true, perform the usual > > > > > lookups. If they succeed, continue with the existing logic. If they > > > > > fail though, don't fail the authentication. Instead, create the > > > > > principal and assign roles directly from the token. > > > > > > > > > > 4) In the Resolver, trust the PolarisPrincipal and avoid re-resolving > > > > > the principal and its roles (as this would fail for detached > > > > > principals). > > > > > > > > > > I believe this change will not impact internal authentication with > > the > > > > > default authorizer. It also won't impact people using external IDPs > > > > > with pre-synchronized federated principals: the default authorizer > > > > > would work just fine. > > > > > > > > > > However, for external IDPs combined with custom authorizers, this > > > > > change would significantly improve the user experience by eliminating > > > > > the need to persist the principal and its roles within Polaris. > > > > > > > > > > What do you all think? > > > > > > > > > > Thanks, > > > > > Alex > > > > > > > > > > > >
