Thanks Cheng Pan! Updating the SPIP with Steve's suggestions.
Parth On Wed, Jul 8, 2026 at 10:42 AM Cheng Pan <[email protected]> wrote: > Thanks Peter and Parth for summarize the discussion, now I’m fine with > this SPIP given you provide several valid cases that OIDC approach does not > offer. > > For HADOOP-19906, on JDK 25, when kerberos is disabled, all threads see > login UGI, thus credential can correctly propagation; when kerberos is > enabled, issue happens. Since the goal of this SPIP is extending DT > framework to non-kerberos cases, it’s not a blocker. But from the user > perspective, a functionality, that works without kerberos, gets broken with > kerberos looks weird. > > Anyway, HADOOP-19906 has landed Hadoop branch-3.5 and will be delivered in > Hadoop 3.5.1 in a few months. > > > Steve's suggestion is a real simplification. We can remove the > DirectTokenProvider trait and just change HadoopDelegationTokenManager. > > +1 for this direction. > > Thanks, > Cheng Pan > > > > On Jul 9, 2026, at 01:00, Parth Chandra <[email protected]> wrote: > > Thanks Peter! You are correct CoarseGrainedExecutorBackend does wrap > SparkHadoopUtil.runAsSparkUser. However, as you pointed out, the end result > does not change. > You are also correct that HADOOP-19906 is an orthogonal issue and must be > addressed. > > Also, Steve's suggestion is a real simplification. We can remove the > DirectTokenProvider trait and just change HadoopDelegationTokenManager. We > sort of lose the explicit signal that the provider does not use Kerberos, > though that is now signalled by the provider indirectly, so we do not lose > much. > > Parth > > On Wed, Jul 8, 2026 at 8:59 AM Peter Toth <[email protected]> wrote: > >> Thanks all for the great discussion. >> >> First, a small correction on the JDK 25 / Subject-propagation point. I >> went through the code and it doesn't seem quite right that "on executor >> task threads there is no active doAs() scope" because >> CoarseGrainedExecutorBackend wraps everything in >> SparkHadoopUtil.runAsSparkUser, which does createSparkUser().doAs(...). >> If I'm not mistaken what happens is that JEP 486 stops that Subject from >> propagating to the RPC/task threads, so getCurrentUser() falls back to the >> static login user, and the pushed credentials are read back from there, >> which is why the outcome Parth described still holds, just via the >> login-user fallback rather than the absence of a doAs. (Parth, please >> correct me if I'm misreading the code here.) >> >> IMO, HADOOP-19906 looks orthogonal to this SPIP rather than a requirement >> of it. It's the same pre-existing JDK 25 dependency that the current >> Kerberos and Kafka delegation-token paths already need. So I'd suggest we >> track it separately and not treat it as a blocker here. >> >> That leaves the main open question: the overlap with the OIDC Credential >> Propagation SPIP. >> Parth listed three cases that the OIDC approach can't cover and that fit >> naturally into the existing delegation-token framework: >> 1. Kafka delegation tokens over SCRAM — they target brokers (not URIs) >> and need no UserContext, so they can't be expressed as >> CredentialProvider.resolve(UserContext, URI). >> 2. The existing S3A/ABFS Hadoop delegation-token bindings — these already >> work under Kerberos and just need the activation gates removed; the OIDC >> manager doesn't run them. >> 3. Proprietary, non-JWT IdP tokens — the OIDC path requires an OIDC JWT. >> >> Cheng Pan (and anyone else who has the redundancy concern), does this >> resolve it for you? My read is that the two are complementary rather than >> redundant: >> - this SPIP unlocks the existing, provider-agnostic DT mechanism for any >> non-Kerberos provider; >> - while the OIDC SPIP adds per-user/session identity propagation for OIDC. >> >> I'd like to make sure we agree on that framing, and that you're >> comfortable with the direction of extending the existing DT management, >> before we go further and discuss the implementation options that Steve >> mentioned earlier. >> >> Best, >> Peter >> >> On Mon, Jun 29, 2026 at 11:58 PM Parth Chandra <[email protected]> wrote: >> >>> Hi Cheng Pan >>> >>> >> then we decided to keep them independent >>> >>> > I see that, but if we decide to accept and implement both SPIPs, then >>> we are going to provide two approaches for users that enable cloud >>> credentials refresh, this is functionality redundant, and as you know, this >>> part usually involve private data and 3rd party services dependencies, when >>> user report issues, they are likely limited to share the related part of >>> logs and environment information to provide a minimal reproducible cases, >>> this makes diagnosis extremely difficult. Offering two distinct cloud >>> credential refresh mechanisms undoubtedly increases system complexity. >>> > I would lean towards to the OIDC Credential Propagation approach >>> unless it does not cover the functionality (user perspective) provided by >>> this SPIP. >>> >>> There are a few cases covered by this not covered by the OIDC >>> credential propagation approach. For instance, - >>> 1. Kafka delegation tokens over SCRAM — the existing >>> KafkaDelegationTokenProvider is currently blocked by the Kerberos gates. It >>> targets brokers, not URIs, and doesn't need a UserContext. However, the >>> OIDC approach depends on a UserContext (an OIDC JWT with >>> principal/issuer/rawToken). It cannot be reimplemented as a >>> CredentialProvider.resolve(UserContext, URI). >>> 2. Existing S3A/ABFS delegation token bindings (Steve's point in this >>> thread) — these are already implemented as HadoopDelegationTokenProvider >>> and work today in Kerberos environments. They just need the activation >>> gates removed to work without Kerberos. The OIDC SPIP's parallel manager >>> does not unblock them. >>> 3. There can also be proprietary IdP systems which have non JWT tokens >>> (which is what prompted this SPIP in the first place) >>> >>> >> The DirectProviderPath proposed in this SPIP does not go >>> through Subject.doAs() or UserGroupInformation.doAs() and will be >>> unaffected. The existing Kerberos path will have to be updated. >>> > Sorry, I overlook this reply. I think this is also affected. The JDK >>> change breaks the Subject propagation between threads, that means you can >>> not get the same Subject (UGI) instance from the task thread as >>> the updateTokensTask thread, so you can not access any kind of the >>> credential you offered from the task thread. >>> >>> On executor task threads, there is no active doAs()/callAs() scope. >>> getCurrentUser() sees null from Subject.current() (or Subject.getSubject() >>> on older JDKs) and falls back to getLoginUser() — which is a static field >>> and is not based on Subject propagation. The credentials added via >>> addCredentials() on the RPC handler thread are added to this same static >>> login user instance so task threads reading from getLoginUser() should see >>> them >>> However, broadly speaking, we do need HADOOP-19906 >>> >>> I've updated the SPIP document to include these points >>> >>> Parth >>> >> >
