Hi, I would like to reinitiate the discussion on Delegation token support for Kafka.
Brief summary of the past discussion: 1) Broker stores delegation tokens in zookeeper. All brokers will have a cache backed by zookeeper so they will all get notified whenever a new token is generated and they will update their local cache whenever token state changes. 2) The current proposal does not support rotation of secret 3) Only allow the renewal by users that authenticated using *non* delegation token mechanism 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can authenticate using SCRAM-SHA-256, providing the delegation token HMAC as password. Updated the KIP with the following: 1. Protocol and Config changes 2. format of the data stored in ZK. 3. Changes to Java Clients/Usage of SASL SCRAM mechanism https://cwiki.apache.org/confluence/display/KAFKA/KIP- 48+Delegation+token+support+for+Kafka Jun, Ashish, Gwen, Pl review the updated KIP. Thanks Manikumar On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <asi...@cloudera.com> wrote: > Harsha/ Gwen, > > How do we proceed here? I am willing to help out with here. > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <g...@confluent.io> wrote: > > > Is it updated? are all concerns addressed? do you want to start a vote? > > > > Sorry for being pushy, I do appreciate that we are all volunteers and > > finding time is difficult. This feature is important for anything that > > integrates with Kafka (stream processors, Flume, NiFi, etc) and I > > don't want to see this getting stuck because we lack coordination > > within the community. > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <ka...@harsha.io> > > wrote: > > > The only pending update for the KIP is to write up the protocol changes > > like > > > we've it KIP-4. I'll update the wiki. > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <asi...@cloudera.com> > > wrote: > > >> > > >> I think we decided to not support secret rotation, I guess this can be > > >> stated clearly on the KIP. Also, more details on how clients will > > perform > > >> token distribution and how CLI will look like will be helpful. > > >> > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <g...@confluent.io> > > wrote: > > >> > > >> > Hi Guys, > > >> > > > >> > This discussion was dead for a while. Are there still contentious > > >> > points? If not, why are there no votes? > > >> > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <j...@confluent.io> wrote: > > >> > > Ashish, > > >> > > > > >> > > Yes, I will send out a KIP invite for next week to discuss KIP-48 > > and > > >> > other > > >> > > remaining KIPs. > > >> > > > > >> > > Thanks, > > >> > > > > >> > > Jun > > >> > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh < > asi...@cloudera.com> > > >> > wrote: > > >> > > > > >> > >> Thanks Harsha! > > >> > >> > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did > > not > > >> > >> actually make a call on when we should have next KIP call. As > there > > >> > >> are > > >> > a > > >> > >> few outstanding KIPs that could not be discussed this week, can > we > > >> > >> have > > >> > a > > >> > >> KIP hangout call next week? > > >> > >> > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani > > >> > >> <ka...@harsha.io> > > >> > >> wrote: > > >> > >> > > >> > >>> Ashish, > > >> > >>> Yes we are working on it. Lets discuss in the next KIP > > >> > >>> meeting. > > >> > >>> I'll join. > > >> > >>> -Harsha > > >> > >>> > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh < > > asi...@cloudera.com> > > >> > >>> wrote: > > >> > >>> > > >> > >>> > Hello Harsha, > > >> > >>> > > > >> > >>> > Are you still working on this? Wondering if we can discuss > this > > in > > >> > next > > >> > >>> KIP > > >> > >>> > meeting, if you can join. > > >> > >>> > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani < > > >> > ka...@harsha.io> > > >> > >>> > wrote: > > >> > >>> > > > >> > >>> > > Hi Grant, > > >> > >>> > > We are working on it. Will add the details to KIP > > >> > >>> > > about > > >> > the > > >> > >>> > > request protocol. > > >> > >>> > > > > >> > >>> > > Thanks, > > >> > >>> > > Harsha > > >> > >>> > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke > > >> > >>> > > <ghe...@cloudera.com> > > >> > >>> wrote: > > >> > >>> > > > > >> > >>> > > > Hi Parth, > > >> > >>> > > > > > >> > >>> > > > Are you still working on this? If you need any help please > > >> > >>> > > > don't > > >> > >>> > hesitate > > >> > >>> > > > to ask. > > >> > >>> > > > > > >> > >>> > > > Thanks, > > >> > >>> > > > Grant > > >> > >>> > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao < > j...@confluent.io> > > >> > wrote: > > >> > >>> > > > > > >> > >>> > > > > Parth, > > >> > >>> > > > > > > >> > >>> > > > > Thanks for the reply. > > >> > >>> > > > > > > >> > >>> > > > > It makes sense to only allow the renewal by users that > > >> > >>> authenticated > > >> > >>> > > > using > > >> > >>> > > > > *non* delegation token mechanism. Then, should we make > the > > >> > >>> renewal a > > >> > >>> > > > list? > > >> > >>> > > > > For example, in the case of rest proxy, it will be > useful > > >> > >>> > > > > for > > >> > >>> every > > >> > >>> > > > > instance of rest proxy to be able to renew the tokens. > > >> > >>> > > > > > > >> > >>> > > > > It would be clearer if we can document the request > > protocol > > >> > like > > >> > >>> > > > > > > >> > >>> > > > > > > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > >> > >>> > > > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+ > > operations#KIP-4- > > >> > >>> > > Commandlineandcentralizedadministrativeoperations- > > >> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0. > > 10.1.0) > > >> > >>> > > > > . > > >> > >>> > > > > > > >> > >>> > > > > It would also be useful to document the client APIs. > > >> > >>> > > > > > > >> > >>> > > > > Thanks, > > >> > >>> > > > > > > >> > >>> > > > > Jun > > >> > >>> > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt < > > >> > >>> > > > > brahmbhatt.pa...@gmail.com> wrote: > > >> > >>> > > > > > > >> > >>> > > > > > Hi, > > >> > >>> > > > > > > > >> > >>> > > > > > I am suggesting that we will only allow the renewal by > > >> > >>> > > > > > users > > >> > >>> that > > >> > >>> > > > > > authenticated using *non* delegation token mechanism. > > For > > >> > >>> example, > > >> > >>> > If > > >> > >>> > > > > user > > >> > >>> > > > > > Alice authenticated using kerberos and requested > > >> > >>> > > > > > delegation > > >> > >>> tokens, > > >> > >>> > > > only > > >> > >>> > > > > > user Alice authenticated via non delegation token > > >> > >>> > > > > > mechanism > > >> > can > > >> > >>> > > renew. > > >> > >>> > > > > > Clients that have access to delegation tokens can not > > >> > >>> > > > > > issue > > >> > >>> > renewal > > >> > >>> > > > > > request for renewing their own token and this is > > primarily > > >> > >>> > important > > >> > >>> > > to > > >> > >>> > > > > > reduce the time window for which a compromised token > > will > > >> > >>> > > > > > be > > >> > >>> valid. > > >> > >>> > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user can request > > >> > >>> > > > > > delegation > > >> > >>> > tokens > > >> > >>> > > > but > > >> > >>> > > > > > even here I would recommend to avoid creating a chain > > >> > >>> > > > > > where a > > >> > >>> > client > > >> > >>> > > > > > authenticated via delegation token request for more > > >> > delegation > > >> > >>> > > tokens. > > >> > >>> > > > > > Basically anyone can request delegation token, as long > > as > > >> > they > > >> > >>> > > > > authenticate > > >> > >>> > > > > > via a non delegation token mechanism. > > >> > >>> > > > > > > > >> > >>> > > > > > Aren't classes listed here > > >> > >>> > > > > > < > > >> > >>> > > > > > > > >> > >>> > > > > > > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > >> > >>> > > 48+Delegation+token+support+fo > r+Kafka#KIP-48Delegationtokens > > >> > >>> upportforKaf > > >> > >>> > > ka-PublicInterfaces > > >> > >>> > > > > > > > > >> > >>> > > > > > sufficient? > > >> > >>> > > > > > > > >> > >>> > > > > > Thanks > > >> > >>> > > > > > Parth > > >> > >>> > > > > > > > >> > >>> > > > > > > > >> > >>> > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao > > >> > >>> > > > > > <j...@confluent.io> > > >> > >>> wrote: > > >> > >>> > > > > > > > >> > >>> > > > > > > Parth, > > >> > >>> > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of comments inline > > below. > > >> > >>> > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt < > > >> > >>> > > > > > > brahmbhatt.pa...@gmail.com> wrote: > > >> > >>> > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original > > requester > > >> > >>> only? or > > >> > >>> > > > using > > >> > >>> > > > > > > > Kerberos > > >> > >>> > > > > > > > auth only? > > >> > >>> > > > > > > > My recommendation is to do this only using > Kerberos > > >> > >>> > > > > > > > auth > > >> > and > > >> > >>> > only > > >> > >>> > > > > threw > > >> > >>> > > > > > > the > > >> > >>> > > > > > > > renewer specified during the acquisition request. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying > that > > >> > >>> > > > > > > any > > >> > >>> client > > >> > >>> > > > > > > authenticated with the delegation token can renew, > > i.e. > > >> > there > > >> > >>> is > > >> > >>> > no > > >> > >>> > > > > > renewer > > >> > >>> > > > > > > needed? > > >> > >>> > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any authenticated client > > (either > > >> > >>> through > > >> > >>> > > SASL > > >> > >>> > > > > or > > >> > >>> > > > > > > SSL) can request a delegation token for the > > >> > >>> > > > > > > authenticated > > >> > >>> user, > > >> > >>> > > > right? > > >> > >>> > > > > > > > > >> > >>> > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK? > > >> > >>> > > > > > > > My recommendation is still to store in ZK or not > > store > > >> > them > > >> > >>> at > > >> > >>> > > all. > > >> > >>> > > > > The > > >> > >>> > > > > > > > whole controller based distribution is too much > > >> > >>> > > > > > > > overhead > > >> > >>> with > > >> > >>> > not > > >> > >>> > > > > much > > >> > >>> > > > > > to > > >> > >>> > > > > > > > achieve. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired? > > >> > >>> > > > > > > > Either by expiration time out or through an > explicit > > >> > >>> request to > > >> > >>> > > > > > > invalidate. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used? > > >> > >>> > > > > > > > SCRAM > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't > in > > >> > >>> > > > > > > > the > > >> > KIP > > >> > >>> but > > >> > >>> > > was > > >> > >>> > > > > > > > discussed > > >> > >>> > > > > > > > in this thread)? > > >> > >>> > > > > > > > There is no imperonation proposal. I tried and > > >> > >>> > > > > > > > explained > > >> > how > > >> > >>> > its > > >> > >>> > > a > > >> > >>> > > > > > > > different problem and why its not really necessary > > to > > >> > >>> discuss > > >> > >>> > > that > > >> > >>> > > > as > > >> > >>> > > > > > > part > > >> > >>> > > > > > > > of this KIP. This KIP will not support any > > >> > impersonation, > > >> > >>> it > > >> > >>> > > will > > >> > >>> > > > > just > > >> > >>> > > > > > > be > > >> > >>> > > > > > > > another way to authenticate. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions? > > >> > >>> > > > > > > > We do not need new ACLs. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > > > >> > >>> > > > > > > Could we document the format of the new > > request/response > > >> > and > > >> > >>> > their > > >> > >>> > > > > > > associated Resource and Operation for ACL? > > >> > >>> > > > > > > > > >> > >>> > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation token be configured in > > the > > >> > >>> client? > > >> > >>> > > > > > > > Should be through config. I wasn't planning on > > >> > >>> > > > > > > > supporting > > >> > >>> JAAS > > >> > >>> > > for > > >> > >>> > > > > > > tokens. > > >> > >>> > > > > > > > I don't believe hadoop does this either. > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > Thanks > > >> > >>> > > > > > > > Parth > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao < > > >> > j...@confluent.io> > > >> > >>> > > wrote: > > >> > >>> > > > > > > > > > >> > >>> > > > > > > > > Harsha, > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > Another question. > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation token be configured > in > > >> > >>> > > > > > > > > the > > >> > >>> > client? > > >> > >>> > > > The > > >> > >>> > > > > > > > standard > > >> > >>> > > > > > > > > way is to do this through JAAS. However, we will > > >> > >>> > > > > > > > > need > > >> > to > > >> > >>> > think > > >> > >>> > > > > > through > > >> > >>> > > > > > > if > > >> > >>> > > > > > > > > this is convenient in a shared environment. For > > >> > example, > > >> > >>> > when a > > >> > >>> > > > new > > >> > >>> > > > > > > task > > >> > >>> > > > > > > > is > > >> > >>> > > > > > > > > added to a Storm worker node, do we need to > > >> > >>> > > > > > > > > dynamically > > >> > >>> add a > > >> > >>> > > new > > >> > >>> > > > > > > section > > >> > >>> > > > > > > > > in the JAAS file? It may be more convenient if > we > > >> > >>> > > > > > > > > can > > >> > >>> pass in > > >> > >>> > > the > > >> > >>> > > > > > token > > >> > >>> > > > > > > > > through the config directly w/o going through > > JAAS. > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still actively working on this > > KIP? > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > Thanks, > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > Jun > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao < > > >> > >>> j...@confluent.io> > > >> > >>> > > > wrote: > > >> > >>> > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list. > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to document the format of > > the > > >> > data > > >> > >>> > stored > > >> > >>> > > > in > > >> > >>> > > > > > ZK. > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether > > the > > >> > tokens > > >> > >>> > > should > > >> > >>> > > > > be > > >> > >>> > > > > > > > > > propagated through ZK like config/acl/quota, > or > > >> > through > > >> > >>> the > > >> > >>> > > > > > > controller. > > >> > >>> > > > > > > > > > Currently, the controller is only designed for > > >> > >>> propagating > > >> > >>> > > > topic > > >> > >>> > > > > > > > > metadata, > > >> > >>> > > > > > > > > > but not other data. > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token > instead > > >> > >>> > > > > > > > > > of > > >> > >>> > > DIGEST-MD5 > > >> > >>> > > > > > since > > >> > >>> > > > > > > > it's > > >> > >>> > > > > > > > > > deprecated? > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem broken. > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks, > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > > Jun > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen > Shapira < > > >> > >>> > > > > g...@confluent.io> > > >> > >>> > > > > > > > > wrote: > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see, remaining questions are: > > >> > >>> > > > > > > > > >> > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original > > >> > requester > > >> > >>> > only? > > >> > >>> > > > or > > >> > >>> > > > > > > using > > >> > >>> > > > > > > > > >> Kerberos auth only? > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK? > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired? > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used? > > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it > > wasn't > > >> > >>> > > > > > > > > >> in > > >> > the > > >> > >>> > KIP > > >> > >>> > > > but > > >> > >>> > > > > > was > > >> > >>> > > > > > > > > >> discussed in this thread)? > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what > > actions? > > >> > >>> > > > > > > > > >> > > >> > >>> > > > > > > > > >> Gwen > > >> > >>> > > > > > > > > >> > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha < > > >> > >>> ka...@harsha.io> > > >> > >>> > > > wrote: > > >> > >>> > > > > > > > > >> > Jun & Ismael, > > >> > >>> > > > > > > > > >> > Unfortunately I > > >> > >>> > > > > > > > > >> > couldn't > > >> > >>> attend > > >> > >>> > > the > > >> > >>> > > > > KIP > > >> > >>> > > > > > > > > meeting > > >> > >>> > > > > > > > > >> > when delegation > > tokens > > >> > >>> > discussed. > > >> > >>> > > > > > > > Appreciate > > >> > >>> > > > > > > > > if > > >> > >>> > > > > > > > > >> > you can update the > > >> > thread if > > >> > >>> > you > > >> > >>> > > > have > > >> > >>> > > > > > any > > >> > >>> > > > > > > > > >> > further questions. > > >> > >>> > > > > > > > > >> > Thanks, > > >> > >>> > > > > > > > > >> > Harsha > > >> > >>> > > > > > > > > >> > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan > Pei > > >> > wrote: > > >> > >>> > > > > > > > > >> >> It seems that the links to images in the > KIP > > >> > >>> > > > > > > > > >> >> are > > >> > >>> > broken. > > >> > >>> > > > > > > > > >> >> > > >> > >>> > > > > > > > > >> >> Liquan > > >> > >>> > > > > > > > > >> >> > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth > > >> > brahmbhatt < > > >> > >>> > > > > > > > > >> >> brahmbhatt.pa...@gmail.com> wrote: > > >> > >>> > > > > > > > > >> >> > > >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs > mean? > > >> > >>> > > > > > > > > >> >> > In the current proposal we only allow a > > user > > >> > >>> > > > > > > > > >> >> > to > > >> > >>> get > > >> > >>> > > > > > delegation > > >> > >>> > > > > > > > > token > > >> > >>> > > > > > > > > >> for > > >> > >>> > > > > > > > > >> >> > the identity that it authenticated as > > using > > >> > >>> another > > >> > >>> > > > > > mechanism, > > >> > >>> > > > > > > > i.e. > > >> > >>> > > > > > > > > >> A user > > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab for > > >> > >>> > > > > > > > > >> >> > principal > > >> > >>> > > > > > > us...@example.com > > >> > >>> > > > > > > > > >> will get > > >> > >>> > > > > > > > > >> >> > delegation tokens for that user only. In > > >> > future I > > >> > >>> > think > > >> > >>> > > > we > > >> > >>> > > > > > will > > >> > >>> > > > > > > > > have > > >> > >>> > > > > > > > > >> to > > >> > >>> > > > > > > > > >> >> > extend support such that we allow some > set > > >> > >>> > > > > > > > > >> >> > of > > >> > >>> users ( > > >> > >>> > > > > > > > > >> >> > kafka-rest-u...@example.com, > > >> > >>> > storm-nim...@example.com) > > >> > >>> > > > to > > >> > >>> > > > > > > > acquire > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other > users > > >> > whose > > >> > >>> > > identity > > >> > >>> > > > > > they > > >> > >>> > > > > > > > have > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers > > will > > >> > have > > >> > >>> ACLs > > >> > >>> > > to > > >> > >>> > > > > > > control > > >> > >>> > > > > > > > > >> which > > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate other > > users > > >> > and > > >> > >>> get > > >> > >>> > > > tokens > > >> > >>> > > > > > on > > >> > >>> > > > > > > > > >> behalf of > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole > > >> > different > > >> > >>> > > problem > > >> > >>> > > > in > > >> > >>> > > > > > my > > >> > >>> > > > > > > > > >> opinion and > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in separate > KIP. > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting > > and > > >> > >>> renewing > > >> > >>> > > > > > delegation > > >> > >>> > > > > > > > > >> tokens? > > >> > >>> > > > > > > > > >> >> > Typically this should be very very low, > 1 > > >> > request > > >> > >>> per > > >> > >>> > > > > minute > > >> > >>> > > > > > > is a > > >> > >>> > > > > > > > > >> >> > relatively high estimate. However it > > depends > > >> > >>> > > > > > > > > >> >> > on > > >> > >>> the > > >> > >>> > > token > > >> > >>> > > > > > > > > >> expiration. I am > > >> > >>> > > > > > > > > >> >> > less worried about the extra load it > puts > > on > > >> > >>> > controller > > >> > >>> > > > vs > > >> > >>> > > > > > the > > >> > >>> > > > > > > > > added > > >> > >>> > > > > > > > > >> >> > complexity and the value it offers. > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > Thanks > > >> > >>> > > > > > > > > >> >> > Parth > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael > > Juma > > >> > >>> > > > > > > > > >> >> > < > > >> > >>> > > > > > > ism...@juma.me.uk> > > >> > >>> > > > > > > > > >> wrote: > > >> > >>> > > > > > > > > >> >> > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably > > require a > > >> > >>> separate > > >> > >>> > > KIP > > >> > >>> > > > > as > > >> > >>> > > > > > it > > >> > >>> > > > > > > > > will > > >> > >>> > > > > > > > > >> >> > > introduce user visible changes. We > could > > >> > >>> > > > > > > > > >> >> > > also > > >> > >>> > update > > >> > >>> > > > > KIP-48 > > >> > >>> > > > > > > to > > >> > >>> > > > > > > > > >> have this > > >> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to > do > > it > > >> > >>> > > separately. > > >> > >>> > > > We > > >> > >>> > > > > > can > > >> > >>> > > > > > > > > >> discuss > > >> > >>> > > > > > > > > >> >> > that > > >> > >>> > > > > > > > > >> >> > > in the KIP call today. > > >> > >>> > > > > > > > > >> >> > > > > >> > >>> > > > > > > > > >> >> > > Ismael > > >> > >>> > > > > > > > > >> >> > > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, > Rajini > > >> > Sivaram > > >> > >>> < > > >> > >>> > > > > > > > > >> >> > > rajinisiva...@googlemail.com> wrote: > > >> > >>> > > > > > > > > >> >> > > > > >> > >>> > > > > > > > > >> >> > > > Ismael, > > >> > >>> > > > > > > > > >> >> > > > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA ( > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/ > > >> > jira/browse/KAFKA-3751) > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL > mechanism. > > >> > >>> > > > > > > > > >> >> > > > Would > > >> > >>> that > > >> > >>> > > need > > >> > >>> > > > > > > another > > >> > >>> > > > > > > > > >> KIP? If > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can > > this > > >> > just > > >> > >>> be > > >> > >>> > a > > >> > >>> > > > JIRA > > >> > >>> > > > > > > that > > >> > >>> > > > > > > > > gets > > >> > >>> > > > > > > > > >> >> > > reviewed > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready? > > >> > >>> > > > > > > > > >> >> > > > > > >> > >>> > > > > > > > > >> >> > > > Thank you, > > >> > >>> > > > > > > > > >> >> > > > > > >> > >>> > > > > > > > > >> >> > > > Rajini > > >> > >>> > > > > > > > > >> >> > > > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, > > Ismael > > >> > Juma < > > >> > >>> > > > > > > > > ism...@juma.me.uk> > > >> > >>> > > > > > > > > >> >> > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a > > good > > >> > >>> > candidate. > > >> > >>> > > > > > > > > >> >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned > > this > > >> > >>> > > > > > > > > >> >> > > > > as > > >> > a > > >> > >>> SASL > > >> > >>> > > > > > mechanism > > >> > >>> > > > > > > > > that > > >> > >>> > > > > > > > > >> might > > >> > >>> > > > > > > > > >> >> > be > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been > > >> > >>> > > > > > > > > >> >> > > > > meaning > > >> > to > > >> > >>> > check > > >> > >>> > > > it > > >> > >>> > > > > in > > >> > >>> > > > > > > > more > > >> > >>> > > > > > > > > >> detail. > > >> > >>> > > > > > > > > >> >> > > Good > > >> > >>> > > > > > > > > >> >> > > > > to know that you are willing to > > >> > contribute > > >> > >>> an > > >> > >>> > > > > > > > implementation. > > >> > >>> > > > > > > > > >> Maybe > > >> > >>> > > > > > > > > >> >> > we > > >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for > > this? > > >> > >>> > > > > > > > > >> >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Ismael > > >> > >>> > > > > > > > > >> >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, > > >> > >>> > > > > > > > > >> >> > > > > Rajini > > >> > >>> > Sivaram < > > >> > >>> > > > > > > > > >> >> > > > > rajinisiva...@googlemail.com> > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response > > >> > >>> > Authentication > > >> > >>> > > > > > > > Mechanism) > > >> > >>> > > > > > > > > >> is a > > >> > >>> > > > > > > > > >> >> > > better > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java > > >> > >>> > > > > > > > > >> >> > > > > > doesn't > > >> > >>> come > > >> > >>> > > > with a > > >> > >>> > > > > > > > > built-in > > >> > >>> > > > > > > > > >> SCRAM > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I > > will > > >> > >>> > > > > > > > > >> >> > > > > > be > > >> > >>> happy > > >> > >>> > > to > > >> > >>> > > > > add > > >> > >>> > > > > > > > > support > > >> > >>> > > > > > > > > >> in > > >> > >>> > > > > > > > > >> >> > Kafka > > >> > >>> > > > > > > > > >> >> > > > > since > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism > to > > >> > support > > >> > >>> > > anyway. > > >> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/ > > rfc7677 > > >> > >>> > describes > > >> > >>> > > > the > > >> > >>> > > > > > > > protocol > > >> > >>> > > > > > > > > >> for > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256. > > >> > >>> > > > > > > > > >> >> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, > > Jun > > >> > Rao < > > >> > >>> > > > > > > > j...@confluent.io > > >> > >>> > > > > > > > > > > > >> > >>> > > > > > > > > >> wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A > > >> > >>> > > > > > > > > >> >> > > > > > > couple > > >> > of > > >> > >>> > more > > >> > >>> > > > > > > questions. > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs > > >> > >>> mean? > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate > of > > >> > getting > > >> > >>> and > > >> > >>> > > > > > renewing > > >> > >>> > > > > > > > > >> delegation > > >> > >>> > > > > > > > > >> >> > > > tokens? > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on > > whether > > >> > they > > >> > >>> > > should > > >> > >>> > > > be > > >> > >>> > > > > > > > > directed > > >> > >>> > > > > > > > > >> to the > > >> > >>> > > > > > > > > >> >> > > > > > > controller. > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 > PM, > > >> > parth > > >> > >>> > > > > brahmbhatt < > > >> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.pa...@gmail.com> > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun, > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster > > action > > >> > >>> > > > > > > > > >> >> > > > > > > > to > > >> > add > > >> > >>> > acls > > >> > >>> > > > on > > >> > >>> > > > > > who > > >> > >>> > > > > > > > can > > >> > >>> > > > > > > > > >> request > > >> > >>> > > > > > > > > >> >> > > > > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use > > case > > >> > for > > >> > >>> that > > >> > >>> > > yet > > >> > >>> > > > > but > > >> > >>> > > > > > > > down > > >> > >>> > > > > > > > > >> the line > > >> > >>> > > > > > > > > >> >> > > > when > > >> > >>> > > > > > > > > >> >> > > > > we > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting > > >> > getDelegationTokenAs > > >> > >>> it > > >> > >>> > > will > > >> > >>> > > > > be > > >> > >>> > > > > > > > > >> necessary. > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to > > be > > >> > only > > >> > >>> > > > > > > used/distributed > > >> > >>> > > > > > > > > >> over > > >> > >>> > > > > > > > > >> >> > secure > > >> > >>> > > > > > > > > >> >> > > > > > > channels. > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design > we > > >> > >>> > > > > > > > > >> >> > > > > > > > end > > >> > up > > >> > >>> > > choosing > > >> > >>> > > > > > > > > >> Invalidation will > > >> > >>> > > > > > > > > >> >> > > be > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every > broker > > >> > >>> > > > > > > > > >> >> > > > > > > > or > > >> > >>> > > controller. > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I > > documented > > >> > >>> somewhere > > >> > >>> > > > that > > >> > >>> > > > > > > > > >> invalidation > > >> > >>> > > > > > > > > >> >> > will > > >> > >>> > > > > > > > > >> >> > > > > > directly > > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but > that > > is > > >> > not > > >> > >>> the > > >> > >>> > > > > intent. > > >> > >>> > > > > > > > > >> Invalidation > > >> > >>> > > > > > > > > >> >> > > will > > >> > >>> > > > > > > > > >> >> > > > > > either > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to > > >> > >>> expiration. No > > >> > >>> > > > > direct > > >> > >>> > > > > > > > > >> zookeeper > > >> > >>> > > > > > > > > >> >> > > > > interaction > > >> > >>> > > > > > > > > >> >> > > > > > > from > > >> > >>> > > > > > > > > >> >> > > > > > > > any client. > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the > > >> > >>> DelegationToken > > >> > >>> > > > > without > > >> > >>> > > > > > > the > > >> > >>> > > > > > > > > >> hmac in > > >> > >>> > > > > > > > > >> >> > the > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about > the > > >> > >>> confusion. > > >> > >>> > > The > > >> > >>> > > > > sole > > >> > >>> > > > > > > > > >> purpose of > > >> > >>> > > > > > > > > >> >> > > > > zookeeper > > >> > >>> > > > > > > > > >> >> > > > > > in > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as > distribution > > >> > channel > > >> > >>> > for > > >> > >>> > > > > tokens > > >> > >>> > > > > > > > > >> between all > > >> > >>> > > > > > > > > >> >> > > > brokers > > >> > >>> > > > > > > > > >> >> > > > > > > and a > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only > tokens > > >> > >>> > > > > > > > > >> >> > > > > > > > that > > >> > >>> were > > >> > >>> > > > > > generated > > >> > >>> > > > > > > by > > >> > >>> > > > > > > > > >> making a > > >> > >>> > > > > > > > > >> >> > > > > request > > >> > >>> > > > > > > > > >> >> > > > > > > to a > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted > (more > > on > > >> > this > > >> > >>> in > > >> > >>> > > > second > > >> > >>> > > > > > > > > >> paragraph). The > > >> > >>> > > > > > > > > >> >> > > > token > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements > > (owner, > > >> > >>> renewer, > > >> > >>> > > > uuid > > >> > >>> > > > > , > > >> > >>> > > > > > > > > >> expiration, > > >> > >>> > > > > > > > > >> >> > > hmac) > > >> > >>> > > > > > > > > >> >> > > > , > > >> > >>> > > > > > > > > >> >> > > > > > one > > >> > >>> > > > > > > > > >> >> > > > > > > of > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally > generated > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac > > >> > >>> but > > >> > >>> > > hmac > > >> > >>> > > > it > > >> > >>> > > > > > > self > > >> > >>> > > > > > > > is > > >> > >>> > > > > > > > > >> >> > derivable > > >> > >>> > > > > > > > > >> >> > > > if > > >> > >>> > > > > > > > > >> >> > > > > > you > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other elements > of > > >> > >>> > > > > > > > > >> >> > > > > > > > the > > >> > >>> token > > >> > >>> > + > > >> > >>> > > > > secret > > >> > >>> > > > > > > key > > >> > >>> > > > > > > > > to > > >> > >>> > > > > > > > > >> >> > generate > > >> > >>> > > > > > > > > >> >> > > > > hmac. > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not > > provide > > >> > SSL > > >> > >>> > > support > > >> > >>> > > > we > > >> > >>> > > > > > do > > >> > >>> > > > > > > > not > > >> > >>> > > > > > > > > >> want the > > >> > >>> > > > > > > > > >> >> > > > > entire > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred > > to > > >> > >>> zookeeper > > >> > >>> > > as > > >> > >>> > > > > that > > >> > >>> > > > > > > > will > > >> > >>> > > > > > > > > >> be an > > >> > >>> > > > > > > > > >> >> > > > insecure > > >> > >>> > > > > > > > > >> >> > > > > > > wire > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only > > store > > >> > >>> > > > > > > > > >> >> > > > > > > > all > > >> > >>> the > > >> > >>> > > other > > >> > >>> > > > > > > > elements > > >> > >>> > > > > > > > > >> of a > > >> > >>> > > > > > > > > >> >> > > > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read > these > > >> > >>> elements > > >> > >>> > and > > >> > >>> > > > > > because > > >> > >>> > > > > > > > > they > > >> > >>> > > > > > > > > >> also > > >> > >>> > > > > > > > > >> >> > > have > > >> > >>> > > > > > > > > >> >> > > > > > access > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be > > able > > >> > >>> > > > > > > > > >> >> > > > > > > > to > > >> > >>> > generate > > >> > >>> > > > > hmac > > >> > >>> > > > > > on > > >> > >>> > > > > > > > > >> their end. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative > > proposed > > >> > >>> > > > > > > > > >> >> > > > > > > > is > > >> > to > > >> > >>> > avoid > > >> > >>> > > > > > > zookeeper > > >> > >>> > > > > > > > > >> >> > > altogether. A > > >> > >>> > > > > > > > > >> >> > > > > > > Client > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with > required > > >> > >>> > information > > >> > >>> > > > > > (owner, > > >> > >>> > > > > > > > > >> renwer, > > >> > >>> > > > > > > > > >> >> > > > > expiration) > > >> > >>> > > > > > > > > >> >> > > > > > > and > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, > uuid). > > >> > Broker > > >> > >>> > won't > > >> > >>> > > > > store > > >> > >>> > > > > > > this > > >> > >>> > > > > > > > > in > > >> > >>> > > > > > > > > >> >> > > zookeeper. > > >> > >>> > > > > > > > > >> >> > > > > > From > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client can > > contact > > >> > >>> > > > > > > > > >> >> > > > > > > > any > > >> > >>> > broker > > >> > >>> > > > with > > >> > >>> > > > > > all > > >> > >>> > > > > > > > the > > >> > >>> > > > > > > > > >> >> > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > token > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, > > expiration, > > >> > hmac, > > >> > >>> > > uuid) > > >> > >>> > > > > the > > >> > >>> > > > > > > > borker > > >> > >>> > > > > > > > > >> will > > >> > >>> > > > > > > > > >> >> > > > > regenerate > > >> > >>> > > > > > > > > >> >> > > > > > > the > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it > matches > > >> > >>> > > > > > > > > >> >> > > > > > > > with > > >> > >>> hmac > > >> > >>> > > > > > presented > > >> > >>> > > > > > > by > > >> > >>> > > > > > > > > >> client , > > >> > >>> > > > > > > > > >> >> > > > broker > > >> > >>> > > > > > > > > >> >> > > > > > > will > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate. > > >> > >>> Only > > >> > >>> > > > > problem > > >> > >>> > > > > > > with > > >> > >>> > > > > > > > > >> this > > >> > >>> > > > > > > > > >> >> > > approach > > >> > >>> > > > > > > > > >> >> > > > > is > > >> > >>> > > > > > > > > >> >> > > > > > if > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is > compromised > > >> > >>> > > > > > > > > >> >> > > > > > > > any > > >> > >>> client > > >> > >>> > > can > > >> > >>> > > > > now > > >> > >>> > > > > > > > > generate > > >> > >>> > > > > > > > > >> >> > random > > >> > >>> > > > > > > > > >> >> > > > > tokens > > >> > >>> > > > > > > > > >> >> > > > > > > and > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to > > >> > >>> authenticate > > >> > >>> > as > > >> > >>> > > > any > > >> > >>> > > > > > user > > >> > >>> > > > > > > > > they > > >> > >>> > > > > > > > > >> like. > > >> > >>> > > > > > > > > >> >> > > with > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that > > only > > >> > >>> tokens > > >> > >>> > > > > acquired > > >> > >>> > > > > > > via > > >> > >>> > > > > > > > a > > >> > >>> > > > > > > > > >> broker > > >> > >>> > > > > > > > > >> >> > > > (using > > >> > >>> > > > > > > > > >> >> > > > > > some > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation > > >> > >>> token) > > >> > >>> > > will > > >> > >>> > > > > be > > >> > >>> > > > > > > > > >> accepted. We > > >> > >>> > > > > > > > > >> >> > > need > > >> > >>> > > > > > > > > >> >> > > > to > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes > > >> > >>> > > > > > > > > >> >> > > > > > > > more > > >> > >>> sense > > >> > >>> > and > > >> > >>> > > > we > > >> > >>> > > > > > can > > >> > >>> > > > > > > go > > >> > >>> > > > > > > > > >> over it > > >> > >>> > > > > > > > > >> >> > in > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the > > invite > > >> > >>> > > > > > > > > >> >> > > > > > > > to > > >> > >>> me? > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at > 10:35 > > >> > >>> > > > > > > > > >> >> > > > > > > > AM, > > >> > Jun > > >> > >>> > Rao < > > >> > >>> > > > > > > > > >> j...@confluent.io> > > >> > >>> > > > > > > > > >> >> > > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few > > >> > comments. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can > be > > >> > useful > > >> > >>> for > > >> > >>> > > > Kafka > > >> > >>> > > > > > > > Connect > > >> > >>> > > > > > > > > >> and > > >> > >>> > > > > > > > > >> >> > Kafka > > >> > >>> > > > > > > > > >> >> > > > > rest > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will > > need > > >> > >>> > > > > > > > > >> >> > > > > > > > > to > > >> > >>> run a > > >> > >>> > > > task > > >> > >>> > > > > on > > >> > >>> > > > > > > > > behalf > > >> > >>> > > > > > > > > >> of a > > >> > >>> > > > > > > > > >> >> > > > client. > > >> > >>> > > > > > > > > >> >> > > > > > We > > >> > >>> > > > > > > > > >> >> > > > > > > > will > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how > > >> > >>> > > > > > > > > >> >> > > > > > > > > those > > >> > >>> > services > > >> > >>> > > > use > > >> > >>> > > > > > > Kafka > > >> > >>> > > > > > > > > >> clients > > >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). > Instead > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a > > >> > >>> > shared > > >> > >>> > > > > client > > >> > >>> > > > > > > per > > >> > >>> > > > > > > > > >> worker, > > >> > >>> > > > > > > > > >> >> > we > > >> > >>> > > > > > > > > >> >> > > > will > > >> > >>> > > > > > > > > >> >> > > > > > > need > > >> > >>> > > > > > > > > >> >> > > > > > > > a > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task since > > the > > >> > >>> > > > authentication > > >> > >>> > > > > > > > happens > > >> > >>> > > > > > > > > >> at the > > >> > >>> > > > > > > > > >> >> > > > > > connection > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, > > the > > >> > >>> renewer > > >> > >>> > > will > > >> > >>> > > > be > > >> > >>> > > > > > the > > >> > >>> > > > > > > > > >> workers. > > >> > >>> > > > > > > > > >> >> > So, > > >> > >>> > > > > > > > > >> >> > > we > > >> > >>> > > > > > > > > >> >> > > > > > > > probably > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers. > > >> > For > > >> > >>> > > Kafka > > >> > >>> > > > > rest > > >> > >>> > > > > > > > > proxy, > > >> > >>> > > > > > > > > >> the > > >> > >>> > > > > > > > > >> >> > > > renewer > > >> > >>> > > > > > > > > >> >> > > > > > can > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the > creator > > >> > >>> > > > > > > > > >> >> > > > > > > > > of > > >> > the > > >> > >>> > > token. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on > > who > > >> > can > > >> > >>> > > request > > >> > >>> > > > > > > > delegation > > >> > >>> > > > > > > > > >> tokens? > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend > people > > to > > >> > send > > >> > >>> > > > > delegation > > >> > >>> > > > > > > > tokens > > >> > >>> > > > > > > > > >> in an > > >> > >>> > > > > > > > > >> >> > > > > encrypted > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel? > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible > for > > >> > expiring > > >> > >>> > > > tokens, > > >> > >>> > > > > > > every > > >> > >>> > > > > > > > > >> broker? > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating > > tokens, > > >> > would > > >> > >>> it > > >> > >>> > be > > >> > >>> > > > > > better > > >> > >>> > > > > > > to > > >> > >>> > > > > > > > > do > > >> > >>> > > > > > > > > >> it in > > >> > >>> > > > > > > > > >> >> > a > > >> > >>> > > > > > > > > >> >> > > > > > request > > >> > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK > > >> > >>> > > > > > > > > >> >> > > > > > > > > directly? > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of > > client > > >> > >>> > > > > > > > > >> >> > > > > > > > > in > > >> > >>> the > > >> > >>> > > wiki > > >> > >>> > > > > > > > sometimes > > >> > >>> > > > > > > > > >> refers > > >> > >>> > > > > > > > > >> >> > to > > >> > >>> > > > > > > > > >> >> > > > the > > >> > >>> > > > > > > > > >> >> > > > > > end > > >> > >>> > > > > > > > > >> >> > > > > > > > > client and some other > times > > >> > refers > > >> > >>> to > > >> > >>> > the > > >> > >>> > > > > > client > > >> > >>> > > > > > > > > using > > >> > >>> > > > > > > > > >> the > > >> > >>> > > > > > > > > >> >> > > > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful > > to > > >> > >>> > distinguish > > >> > >>> > > > > > between > > >> > >>> > > > > > > > the > > >> > >>> > > > > > > > > >> two. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the > > >> > sentence > > >> > >>> > > "Broker > > >> > >>> > > > > > also > > >> > >>> > > > > > > > > >> stores the > > >> > >>> > > > > > > > > >> >> > > > > > > > DelegationToken > > >> > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the > > >> > zookeeper." > > >> > >>> a > > >> > >>> > bit > > >> > >>> > > > > > more? I > > >> > >>> > > > > > > > > >> thought the > > >> > >>> > > > > > > > > >> >> > > > > > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks, > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Jun > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at > 9:22 > > >> > >>> > > > > > > > > >> >> > > > > > > > > AM, > > >> > Jun > > >> > >>> > Rao > > >> > >>> > > < > > >> > >>> > > > > > > > > >> j...@confluent.io> > > >> > >>> > > > > > > > > >> >> > > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha, > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP > > meeting > > >> > >>> invite. > > >> > >>> > We > > >> > >>> > > > can > > >> > >>> > > > > > > > discuss > > >> > >>> > > > > > > > > >> this in > > >> > >>> > > > > > > > > >> >> > > the > > >> > >>> > > > > > > > > >> >> > > > > > > meeting > > >> > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow. > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > Thanks, > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > Jun > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at > > 8:47 > > >> > AM, > > >> > >>> > > Harsha < > > >> > >>> > > > > > > > > >> ka...@harsha.io> > > >> > >>> > > > > > > > > >> >> > > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have > a > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> KIP > > >> > >>> meeting > > >> > >>> > > > > around > > >> > >>> > > > > > > > this. > > >> > >>> > > > > > > > > >> The KIP > > >> > >>> > > > > > > > > >> >> > is > > >> > >>> > > > > > > > > >> >> > > > up > > >> > >>> > > > > > > > > >> >> > > > > > for > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and > > if > > >> > there > > >> > >>> are > > >> > >>> > > any > > >> > >>> > > > > > > > questions > > >> > >>> > > > > > > > > >> lets > > >> > >>> > > > > > > > > >> >> > > > quickly > > >> > >>> > > > > > > > > >> >> > > > > > hash > > >> > >>> > > > > > > > > >> >> > > > > > > > out > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> details. > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, > at > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> 08:40 > > >> > >>> AM, > > >> > >>> > > parth > > >> > >>> > > > > > > > > brahmbhatt > > >> > >>> > > > > > > > > >> wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the > hadoop > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > echo > > >> > >>> > system > > >> > >>> > > > uses > > >> > >>> > > > > > so > > >> > >>> > > > > > > no > > >> > >>> > > > > > > > > >> good > > >> > >>> > > > > > > > > >> >> > reason > > >> > >>> > > > > > > > > >> >> > > > > > really. > > >> > >>> > > > > > > > > >> >> > > > > > > > We > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > could > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever > > is > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > the > > >> > >>> > newest > > >> > >>> > > > > > > > recommended > > >> > >>> > > > > > > > > >> standard > > >> > >>> > > > > > > > > >> >> > > is. > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 > at > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > 3:33 > > >> > >>> AM, > > >> > >>> > > > Ismael > > >> > >>> > > > > > > Juma < > > >> > >>> > > > > > > > > >> >> > > > > ism...@juma.me.uk > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the > KIP. I > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > only > > >> > >>> > started > > >> > >>> > > > > > > reviewing > > >> > >>> > > > > > > > > >> this and > > >> > >>> > > > > > > > > >> >> > > may > > >> > >>> > > > > > > > > >> >> > > > > have > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> additional > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. > The > > >> > >>> immediate > > >> > >>> > > > > question > > >> > >>> > > > > > > that > > >> > >>> > > > > > > > > >> came to > > >> > >>> > > > > > > > > >> >> > > mind > > >> > >>> > > > > > > > > >> >> > > > is > > >> > >>> > > > > > > > > >> >> > > > > > our > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> choice of > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > though > > >> > it's > > >> > >>> > > marked > > >> > >>> > > > > as > > >> > >>> > > > > > > > > >> OBSOLETE in > > >> > >>> > > > > > > > > >> >> > the > > >> > >>> > > > > > > > > >> >> > > > IANA > > >> > >>> > > > > > > > > >> >> > > > > > > > > Registry > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> of > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and > > the > > >> > >>> original > > >> > >>> > > RFC > > >> > >>> > > > > > > (2831) > > >> > >>> > > > > > > > > has > > >> > >>> > > > > > > > > >> been > > >> > >>> > > > > > > > > >> >> > > moved > > >> > >>> > > > > > > > > >> >> > > > > to > > >> > >>> > > > > > > > > >> >> > > > > > > > > Historic > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > status: > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > https://tools.ietf.org/html/ > > >> > >>> > > rfc6331 > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > >> > >>> > > > > > > > > >> > > >> > >>> > > > > > > > > >> > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl- > > >> > >>> mechanisms.xhtml > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the > reasoning > > >> > behind > > >> > >>> > that > > >> > >>> > > > > > choice? > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, > 2016 > > at > > >> > 11:29 > > >> > >>> > PM, > > >> > >>> > > > Gwen > > >> > >>> > > > > > > > > Shapira < > > >> > >>> > > > > > > > > >> >> > > > > > > g...@confluent.io > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> wrote: > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments > > inline > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > :) > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > emphasize > > >> > >>> that > > >> > >>> > > even > > >> > >>> > > > > > though > > >> > >>> > > > > > > > > >> delegation > > >> > >>> > > > > > > > > >> >> > > > tokens > > >> > >>> > > > > > > > > >> >> > > > > > > are a > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I > > feel > > >> > very > > >> > >>> > > strongly > > >> > >>> > > > > > about > > >> > >>> > > > > > > > not > > >> > >>> > > > > > > > > >> adding > > >> > >>> > > > > > > > > >> >> > > > > > dependency > > >> > >>> > > > > > > > > >> >> > > > > > > > on > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when > implementing > > >> > >>> delegation > > >> > >>> > > > > tokens > > >> > >>> > > > > > > for > > >> > >>> > > > > > > > > >> Kafka. The > > >> > >>> > > > > > > > > >> >> > > KIP > > >> > >>> > > > > > > > > >> >> > > > > > > doesn't > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> imply > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such > dependency, > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but > > >> > if > > >> > >>> you > > >> > >>> > > can > > >> > >>> > > > > > > > clarify... > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop > > >> > dependency.* > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add > this > > to > > >> > the > > >> > >>> KIP > > >> > >>> > so > > >> > >>> > > > no > > >> > >>> > > > > > one > > >> > >>> > > > > > > > will > > >> > >>> > > > > > > > > >> read > > >> > >>> > > > > > > > > >> >> > the > > >> > >>> > > > > > > > > >> >> > > > KIP > > >> > >>> > > > > > > > > >> >> > > > > > and > > >> > >>> > > > > > > > > >> >> > > > > > > > > panic > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks > before > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > the > > >> > next > > >> > >>> > > > > release... > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get > > >> > delegation > > >> > >>> > token > > >> > >>> > > at > > >> > >>> > > > > any > > >> > >>> > > > > > > > time > > >> > >>> > > > > > > > > >> after > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticating? > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> only > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately > > after? > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you > > are > > >> > >>> > > > authenticated > > >> > >>> > > > > > you > > >> > >>> > > > > > > > can > > >> > >>> > > > > > > > > >> get > > >> > >>> > > > > > > > > >> >> > > > delegation > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> We > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > need > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a > > client > > >> > >>> > > > authenticated > > >> > >>> > > > > > > using > > >> > >>> > > > > > > > > >> delegation > > >> > >>> > > > > > > > > >> >> > > > > token, > > >> > >>> > > > > > > > > >> >> > > > > > > can > > >> > >>> > > > > > > > > >> >> > > > > > > > > also > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation > token > > >> > again or > > >> > >>> > not. > > >> > >>> > > > > Also > > >> > >>> > > > > > > > there > > >> > >>> > > > > > > > > >> is the > > >> > >>> > > > > > > > > >> >> > > > > question > > >> > >>> > > > > > > > > >> >> > > > > > of > > >> > >>> > > > > > > > > >> >> > > > > > > > do > > >> > >>> > > > > > > > > >> >> > > > > > > > > we > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to > acquire > > >> > >>> delegation > > >> > >>> > > > token > > >> > >>> > > > > > or > > >> > >>> > > > > > > we > > >> > >>> > > > > > > > > >> want > > >> > >>> > > > > > > > > >> >> > > specific > > >> > >>> > > > > > > > > >> > > > > > > > > -- > > Gwen Shapira > > Product Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > > > > > -- > > Regards, > Ashish >
