110. What does getDelegationTokenAs mean? In the current proposal we only allow a user to get delegation token for the identity that it authenticated as using another mechanism, i.e. A user that authenticate using a keytab for principal us...@example.com will get delegation tokens for that user only. In future I think we will have to extend support such that we allow some set of users ( kafka-rest-u...@example.com, storm-nim...@example.com) to acquire delegation tokens on behalf of other users whose identity they have verified independently. Kafka brokers will have ACLs to control which users are allowed to impersonate other users and get tokens on behalf of them. Overall Impersonation is a whole different problem in my opinion and I think we can tackle it in separate KIP.
111. What's the typical rate of getting and renewing delegation tokens? Typically this should be very very low, 1 request per minute is a relatively high estimate. However it depends on the token expiration. I am less worried about the extra load it puts on controller vs the added complexity and the value it offers. Thanks Parth On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <ism...@juma.me.uk> wrote: > Thanks Rajini. It would probably require a separate KIP as it will > introduce user visible changes. We could also update KIP-48 to have this > information, but it seems cleaner to do it separately. We can discuss that > in the KIP call today. > > Ismael > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram < > rajinisiva...@googlemail.com> wrote: > > > Ismael, > > > > I have created a JIRA (https://issues.apache.org/jira/browse/KAFKA-3751) > > for adding SCRAM as a SASL mechanism. Would that need another KIP? If > > KIP-48 will use this mechanism, can this just be a JIRA that gets > reviewed > > when the PR is ready? > > > > Thank you, > > > > Rajini > > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > > > Thanks Rajini, SCRAM seems like a good candidate. > > > > > > Gwen had independently mentioned this as a SASL mechanism that might be > > > useful for Kafka and I have been meaning to check it in more detail. > Good > > > to know that you are willing to contribute an implementation. Maybe we > > > should file a separate JIRA for this? > > > > > > Ismael > > > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram < > > > rajinisiva...@googlemail.com> wrote: > > > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is a > better > > > > mechanism than Digest-MD5. Java doesn't come with a built-in SCRAM > > > > SaslServer or SaslClient, but I will be happy to add support in Kafka > > > since > > > > it would be a useful mechanism to support anyway. > > > > https://tools.ietf.org/html/rfc7677 describes the protocol for > > > > SCRAM-SHA-256. > > > > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <j...@confluent.io> wrote: > > > > > > > > > Parth, > > > > > > > > > > Thanks for the explanation. A couple of more questions. > > > > > > > > > > 110. What does getDelegationTokenAs mean? > > > > > > > > > > 111. What's the typical rate of getting and renewing delegation > > tokens? > > > > > That may have an impact on whether they should be directed to the > > > > > controller. > > > > > > > > > > Jun > > > > > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt < > > > > > brahmbhatt.pa...@gmail.com> wrote: > > > > > > > > > > > Hi Jun, > > > > > > > > > > > > Thanks for reviewing. > > > > > > > > > > > > * We could add a Cluster action to add acls on who can request > > > > delegation > > > > > > tokens. I don't see the use case for that yet but down the line > > when > > > we > > > > > > start supporting getDelegationTokenAs it will be necessary. > > > > > > * Yes we recommend tokens to be only used/distributed over secure > > > > > channels. > > > > > > * Depending on what design we end up choosing Invalidation will > be > > > > > > responsibility of every broker or controller. > > > > > > * I am not sure if I documented somewhere that invalidation will > > > > directly > > > > > > go through zookeeper but that is not the intent. Invalidation > will > > > > either > > > > > > be request based or due to expiration. No direct zookeeper > > > interaction > > > > > from > > > > > > any client. > > > > > > * "Broker also stores the DelegationToken without the hmac in the > > > > > > zookeeper." : Sorry about the confusion. The sole purpose of > > > zookeeper > > > > in > > > > > > this design is as distribution channel for tokens between all > > brokers > > > > > and a > > > > > > layer that ensures only tokens that were generated by making a > > > request > > > > > to a > > > > > > broker will be accepted (more on this in second paragraph). The > > token > > > > > > consists of few elements (owner, renewer, uuid , expiration, > hmac) > > , > > > > one > > > > > of > > > > > > which is the finally generated hmac but hmac it self is derivable > > if > > > > you > > > > > > have all the other elements of the token + secret key to generate > > > hmac. > > > > > > Given zookeeper does not provide SSL support we do not want the > > > entire > > > > > > token to be wire transferred to zookeeper as that will be an > > insecure > > > > > wire > > > > > > transfer. Instead we only store all the other elements of a > > > delegation > > > > > > tokens. Brokers can read these elements and because they also > have > > > > access > > > > > > to secret key they will be able to generate hmac on their end. > > > > > > > > > > > > One of the alternative proposed is to avoid zookeeper > altogether. A > > > > > Client > > > > > > will call broker with required information (owner, renwer, > > > expiration) > > > > > and > > > > > > get back (signed hmac, uuid). Broker won't store this in > zookeeper. > > > > From > > > > > > this point a client can contact any broker with all the > delegation > > > > token > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker will > > > regenerate > > > > > the > > > > > > hmac and as long as it matches with hmac presented by client , > > broker > > > > > will > > > > > > allow the request to authenticate. Only problem with this > approach > > > is > > > > if > > > > > > the secret key is compromised any client can now generate random > > > tokens > > > > > and > > > > > > they will still be able to authenticate as any user they like. > with > > > > > > zookeeper we guarantee that only tokens acquired via a broker > > (using > > > > some > > > > > > auth scheme other than delegation token) will be accepted. We > need > > to > > > > > > discuss which proposal makes more sense and we can go over it in > > > > > tomorrow's > > > > > > meeting. > > > > > > > > > > > > Also, can you forward the invite to me? > > > > > > > > > > > > Thanks > > > > > > Parth > > > > > > > > > > > > > > > > > > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <j...@confluent.io> > > wrote: > > > > > > > > > > > > > Thanks for the KIP. A few comments. > > > > > > > > > > > > > > 100. This potentially can be useful for Kafka Connect and Kafka > > > rest > > > > > > proxy > > > > > > > where a worker agent will need to run a task on behalf of a > > client. > > > > We > > > > > > will > > > > > > > likely need to change how those services use Kafka clients > > > > > > > (producer/consumer). Instead of a shared client per worker, we > > will > > > > > need > > > > > > a > > > > > > > client per user task since the authentication happens at the > > > > connection > > > > > > > level. For Kafka Connect, the renewer will be the workers. So, > we > > > > > > probably > > > > > > > need to allow multiple renewers. For Kafka rest proxy, the > > renewer > > > > can > > > > > > > probably just be the creator of the token. > > > > > > > > > > > > > > 101. Do we need new acl on who can request delegation tokens? > > > > > > > > > > > > > > 102. Do we recommend people to send delegation tokens in an > > > encrypted > > > > > > > channel? > > > > > > > > > > > > > > 103. Who is responsible for expiring tokens, every broker? > > > > > > > > > > > > > > 104. For invalidating tokens, would it be better to do it in a > > > > request > > > > > > > instead of going to ZK directly? > > > > > > > > > > > > > > 105. The terminology of client in the wiki sometimes refers to > > the > > > > end > > > > > > > client and some other times refers to the client using the > > > delegation > > > > > > > tokens. It would be useful to distinguish between the two. > > > > > > > > > > > > > > 106. Could you explain the sentence "Broker also stores the > > > > > > DelegationToken > > > > > > > without the hmac in the zookeeper." a bit more? I thought the > > > > > delegation > > > > > > > token is the hmac. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <j...@confluent.io> > > wrote: > > > > > > > > > > > > > > > Hi, Harsha, > > > > > > > > > > > > > > > > Just sent out a KIP meeting invite. We can discuss this in > the > > > > > meeting > > > > > > > > tomorrow. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <ka...@harsha.io> > > wrote: > > > > > > > > > > > > > > > >> Hi All, > > > > > > > >> Can we have a KIP meeting around this. The KIP is > > up > > > > for > > > > > > > >> sometime and if there are any questions lets > > quickly > > > > hash > > > > > > out > > > > > > > >> details. > > > > > > > >> > > > > > > > >> Thanks, > > > > > > > >> Harsha > > > > > > > >> > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt wrote: > > > > > > > >> > That is what the hadoop echo system uses so no good reason > > > > really. > > > > > > We > > > > > > > >> > could > > > > > > > >> > change it to whatever is the newest recommended standard > is. > > > > > > > >> > > > > > > > > >> > Thanks > > > > > > > >> > Parth > > > > > > > >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma < > > > ism...@juma.me.uk > > > > > > > > > > > > wrote: > > > > > > > >> > > > > > > > > >> > > Hi Parth, > > > > > > > >> > > > > > > > > > >> > > Thanks for the KIP. I only started reviewing this and > may > > > have > > > > > > > >> additional > > > > > > > >> > > questions later. The immediate question that came to > mind > > is > > > > our > > > > > > > >> choice of > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE in the > > IANA > > > > > > > Registry > > > > > > > >> of > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has been > moved > > > to > > > > > > > Historic > > > > > > > >> > > status: > > > > > > > >> > > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331 > > > > > > > >> > > > > > > > > > > > > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml > > > > > > > >> > > > > > > > > > >> > > What is the reasoning behind that choice? > > > > > > > >> > > > > > > > > > >> > > Thanks, > > > > > > > >> > > Ismael > > > > > > > >> > > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira < > > > > > g...@confluent.io > > > > > > > > > > > > > > >> wrote: > > > > > > > >> > > > > > > > > > >> > > > Also comments inline :) > > > > > > > >> > > > > > > > > > > >> > > > > * I want to emphasize that even though delegation > > tokens > > > > > are a > > > > > > > >> Hadoop > > > > > > > >> > > > > innovation, I feel very strongly about not adding > > > > dependency > > > > > > on > > > > > > > >> Hadoop > > > > > > > >> > > > > when implementing delegation tokens for Kafka. The > KIP > > > > > doesn't > > > > > > > >> imply > > > > > > > >> > > > > such dependency, but if you can clarify... > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > *No hadoop dependency.* > > > > > > > >> > > > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will read the > > KIP > > > > and > > > > > > > panic > > > > > > > >> > > > three weeks before the next release... > > > > > > > >> > > > > > > > > > > >> > > > > * Can we get delegation token at any time after > > > > > > authenticating? > > > > > > > >> only > > > > > > > >> > > > > immediately after? > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > *As long as you are authenticated you can get > > delegation > > > > > > tokens. > > > > > > > >> We > > > > > > > >> > > need > > > > > > > >> > > > to > > > > > > > >> > > > > discuss if a client authenticated using delegation > > > token, > > > > > can > > > > > > > also > > > > > > > >> > > > acquire > > > > > > > >> > > > > delegation token again or not. Also there is the > > > question > > > > of > > > > > > do > > > > > > > we > > > > > > > >> > > allow > > > > > > > >> > > > > anyone to acquire delegation token or we want > specific > > > > ACLs > > > > > (I > > > > > > > >> think > > > > > > > >> > > its > > > > > > > >> > > > an > > > > > > > >> > > > > overkill.)* > > > > > > > >> > > > > > > > > > > >> > > > I agree that ACLs is an overkill. > > > > > > > >> > > > > > > > > > > >> > > > I think we are debating two options: Either require > > > Kerberos > > > > > > auth > > > > > > > >> for > > > > > > > >> > > > renewal or require non-owners to renew. > > > > > > > >> > > > I *think* the latter is simpler (it basically require > a > > > "job > > > > > > > master" > > > > > > > >> > > > to take responsibility for the renewal, it will have > its > > > own > > > > > > > >> identity > > > > > > > >> > > > anyway and I think this is the correct design pattern > > > > anyway. > > > > > > For > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate renewals?), but > > it > > > is > > > > > > hard > > > > > > > to > > > > > > > >> > > > debate simplicity without looking at the code changes > > > > > required. > > > > > > If > > > > > > > >> you > > > > > > > >> > > > have a draft of how the "require Kerberos" will look > in > > > > Kafka > > > > > > > code, > > > > > > > >> > > > I'll be happy to take a look. > > > > > > > >> > > > > > > > > > > >> > > > > * My understanding is that tokens will propagate via > > ZK > > > > but > > > > > > > >> without > > > > > > > >> > > > > additional changes to UpdateMetadata protocol, > > correct? > > > > > > Clients > > > > > > > >> > > > > currently don't retry on SASL auth failure (IIRC), > but > > > > since > > > > > > the > > > > > > > >> > > > > tokens propagate between brokers asynch, we will > need > > to > > > > > > retry a > > > > > > > >> bit > > > > > > > >> > > > > to avoid clients failing auth due to timing issues. > > > > > > > >> > > > > > > > > > > > >> > > > > *I am considering 2 alternatives right now. The > > current > > > > > > > documented > > > > > > > >> > > > approach > > > > > > > >> > > > > is zookeeper based and it does not require any > changes > > > to > > > > > > > >> > > UpdateMetadata > > > > > > > >> > > > > protocol. An alternative approach can remove > zookeeper > > > > > > > dependency > > > > > > > >> as > > > > > > > >> > > well > > > > > > > >> > > > > but we can discuss that in KIP discussion call.* > > > > > > > >> > > > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping Jun to > > > > arrange a > > > > > > > call? > > > > > > > >> > > > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just the > > > > controller > > > > > > > issue > > > > > > > >> the > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared secret. > > Not > > > > > sure > > > > > > if > > > > > > > >> we > > > > > > > >> > > > > want to continue the discussion here or on the > wiki. I > > > > think > > > > > > > that > > > > > > > >> we > > > > > > > >> > > > > can decouple the problem of "token distribution" > from > > > > > "shared > > > > > > > >> secret > > > > > > > >> > > > > distribution" and use the controller as the only > token > > > > > > generator > > > > > > > >> to > > > > > > > >> > > > > solve the second issue, while still using ZK async > to > > > > > > distribute > > > > > > > >> > > > > tokens. > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > *As mentioned in the previous Email I am fine with > > that > > > > > > approach > > > > > > > >> as > > > > > > > >> > > long > > > > > > > >> > > > as > > > > > > > >> > > > > we agree that the extra complexity of > adding/updating > > > APIS > > > > > > adds > > > > > > > >> enough > > > > > > > >> > > > > value. The advantage with the controller approach is > > > > secret > > > > > > > >> rotation > > > > > > > >> > > can > > > > > > > >> > > > be > > > > > > > >> > > > > automated,frequent and would not require > deployment. * > > > > > > > >> > > > > > > > > > > >> > > > Can you detail the extra complexity (or point me to > the > > > > email > > > > > I > > > > > > > >> > > > missed?) - which Apis are required? > > > > > > > >> > > > As far as I can tell, clients can already find the > > > > controller > > > > > > from > > > > > > > >> > > > metadata. I'm a bit more concerned about controller > > load. > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > >> > > > > * While I like the idea of forcing kerberos auth for > > > > > renwal, I > > > > > > > >> think > > > > > > > >> > > > > it mixes the transport layer the the request content > > in > > > a > > > > > > pretty > > > > > > > >> ugly > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is > better. > > > > > > > >> > > > > > > > > > > > >> > > > > *I feel this is a necessary evil. While this will > make > > > the > > > > > > kafka > > > > > > > >> code > > > > > > > >> > > > > pretty straight forward , forcing renewer to > > non-owner > > > > > pushes > > > > > > > >> the code > > > > > > > >> > > > > ugliness to client and makes it even harder to > > > > integrate. * > > > > > > > >> > > > > > > > > > > >> > > > As mentioned before, I don't think the "renewal by > > other" > > > > > > approach > > > > > > > >> is > > > > > > > >> > > > that ugly for the clients we expect to use delegation > > > tokens > > > > > > since > > > > > > > >> > > > they will have an app-master of some sort who > requested > > > the > > > > > > token > > > > > > > to > > > > > > > >> > > > begin with. > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > >> > > > > The response for my question on how multiple > > identities > > > > will > > > > > > be > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we don't > > > > > > authenticate > > > > > > > >> each > > > > > > > >> > > > > request, we authenticate connections. > > > > > > > >> > > > > > > > > > > > >> > > > > *We authenticate connections, and only when they are > > > being > > > > > > > >> established. > > > > > > > >> > > > Let > > > > > > > >> > > > > me try to phrase this as a question, in absence of > > > > > delegation > > > > > > > >> tokens if > > > > > > > >> > > > we > > > > > > > >> > > > > had to support the use case using user TGT's how > would > > > we > > > > do > > > > > > it? > > > > > > > >> My > > > > > > > >> > > point > > > > > > > >> > > > > was it would be no different with delegation tokens. > > The > > > > use > > > > > > > case > > > > > > > >> you > > > > > > > >> > > are > > > > > > > >> > > > > describing seems more like impersonation.* > > > > > > > >> > > > > > > > > > > >> > > > Yeah, I thought that one of the things that delegation > > > > tokens > > > > > > > >> handled. > > > > > > > >> > > > Maybe I got it wrong :) > > > > > > > >> > > > > > > > > > > >> > > > Thanks for the detailed answers. > > > > > > > >> > > > > > > > > > > >> > > > Gwen > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks > > > > > > > >> > > > > Parth > > > > > > > >> > > > > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen Shapira < > > > > > > > g...@confluent.io > > > > > > > >> > > > > > > > > >> > > > wrote: > > > > > > > >> > > > > > > > > > > > >> > > > >> Hi Parth and Harsha, > > > > > > > >> > > > >> > > > > > > > >> > > > >> Few more comments: > > > > > > > >> > > > >> > > > > > > > >> > > > >> * The API / RequestResponse section doesn't seem to > > > have > > > > > good > > > > > > > >> > > > >> description of the changes to the Kafka Protocol. > > > Sounds > > > > > like > > > > > > > >> you are > > > > > > > >> > > > >> proposing new DelegationTokenRequest and > > > > RenewTokenRequest > > > > > > (and > > > > > > > >> > > > >> matching responses), without detailing the contents > > of > > > > the > > > > > > > >> requests > > > > > > > >> > > > >> and responses? Or rather, you show the class > > interface, > > > > but > > > > > > not > > > > > > > >> the > > > > > > > >> > > > >> underlying protocol. This could be seen as an > > > > > implementation > > > > > > > >> detail, > > > > > > > >> > > > >> but since the binary protocol is what we provide to > > > > > non-Java > > > > > > > >> clients, > > > > > > > >> > > > >> we need to show the changes there. > > > > > > > >> > > > >> > > > > > > > >> > > > >> * getDelegationToken sounds like > > > > > > delegationTokenRequestHandler? > > > > > > > >> Is it > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client? Its > > > unclear... > > > > > > > >> > > > >> > > > > > > > >> > > > >> * I want to emphasize that even though delegation > > > tokens > > > > > are > > > > > > a > > > > > > > >> Hadoop > > > > > > > >> > > > >> innovation, I feel very strongly about not adding > > > > > dependency > > > > > > on > > > > > > > >> Hadoop > > > > > > > >> > > > >> when implementing delegation tokens for Kafka. The > > KIP > > > > > > doesn't > > > > > > > >> imply > > > > > > > >> > > > >> such dependency, but if you can clarify... > > > > > > > >> > > > >> > > > > > > > >> > > > >> * Can we get delegation token at any time after > > > > > > authenticating? > > > > > > > >> only > > > > > > > >> > > > >> immediately after? > > > > > > > >> > > > >> > > > > > > > >> > > > >> * My understanding is that tokens will propagate > via > > ZK > > > > but > > > > > > > >> without > > > > > > > >> > > > >> additional changes to UpdateMetadata protocol, > > correct? > > > > > > Clients > > > > > > > >> > > > >> currently don't retry on SASL auth failure (IIRC), > > but > > > > > since > > > > > > > the > > > > > > > >> > > > >> tokens propagate between brokers asynch, we will > need > > > to > > > > > > retry > > > > > > > a > > > > > > > >> bit > > > > > > > >> > > > >> to avoid clients failing auth due to timing issues. > > > > > > > >> > > > >> > > > > > > > >> > > > >> * Strongly agreeing on clients not touching ZK > > directly > > > > :) > > > > > > > >> > > > >> > > > > > > > >> > > > >> * I liked Ashish's suggestion of having just the > > > > controller > > > > > > > >> issue the > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared > secret. > > > Not > > > > > sure > > > > > > > if > > > > > > > >> we > > > > > > > >> > > > >> want to continue the discussion here or on the > wiki. > > I > > > > > think > > > > > > > >> that we > > > > > > > >> > > > >> can decouple the problem of "token distribution" > from > > > > > "shared > > > > > > > >> secret > > > > > > > >> > > > >> distribution" and use the controller as the only > > token > > > > > > > generator > > > > > > > >> to > > > > > > > >> > > > >> solve the second issue, while still using ZK async > to > > > > > > > distribute > > > > > > > >> > > > >> tokens. > > > > > > > >> > > > >> > > > > > > > >> > > > >> * I am also uncomfortable with infinite lifetime of > > > > tokens > > > > > > (and > > > > > > > >> hoped > > > > > > > >> > > > >> to hear from others in the community) - but having > > the > > > > > option > > > > > > > and > > > > > > > >> > > > >> documenting it as less secure, allows users to > > > configure > > > > > > their > > > > > > > >> system > > > > > > > >> > > > >> as the wish. > > > > > > > >> > > > >> > > > > > > > >> > > > >> * While I like the idea of forcing kerberos auth > for > > > > > renwal, > > > > > > I > > > > > > > >> think > > > > > > > >> > > > >> it mixes the transport layer the the request > content > > > in a > > > > > > > pretty > > > > > > > >> ugly > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner is > better. > > > > > > > >> > > > >> > > > > > > > >> > > > >> Things I'd still like to see: > > > > > > > >> > > > >> > > > > > > > >> > > > >> * More detailed explanation on what we plan to do > for > > > the > > > > > > java > > > > > > > >> clients > > > > > > > >> > > > >> specifically - new configuration? new APIs? > > > > > > > >> > > > >> The response for my question on how multiple > > identities > > > > > will > > > > > > be > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we don't > > > > > > authenticate > > > > > > > >> each > > > > > > > >> > > > >> request, we authenticate connections. > > > > > > > >> > > > >> > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only used in > > the > > > > > Hadoop > > > > > > > >> > > > >> ecosystem. I'm wondering if there are alternatives > in > > > > other > > > > > > > >> ecosystems > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether there are > > some > > > > > > > >> advantages > > > > > > > >> > > > >> there. > > > > > > > >> > > > >> > > > > > > > >> > > > >> Gwen > > > > > > > >> > > > >> > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha < > > > ka...@harsha.io > > > > > > > > > > > > wrote: > > > > > > > >> > > > >> > Hi Gwen, > > > > > > > >> > > > >> > Can you look at Parth's last reply. > Does > > > it > > > > > > answer > > > > > > > >> your > > > > > > > >> > > > >> > concerns. > > > > > > > >> > > > >> > > > > > > > > >> > > > >> > Thanks, > > > > > > > >> > > > >> > Harsha > > > > > > > >> > > > >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth > brahmbhatt > > > > wrote: > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki already has > > > > details > > > > > on > > > > > > > >> token > > > > > > > >> > > > >> >> expiration > > > > > > > >> > > > >> >> under token acquisition process > > > > > > > >> > > > >> >> < > > > > > > > >> > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition > > > > > > > >> > > > >> >. > > > > > > > >> > > > >> >> Current proposal is that tokens will expire > based > > > on a > > > > > > > server > > > > > > > >> side > > > > > > > >> > > > >> >> configuration (default 24 hours) unless renewed. > > > > Renewal > > > > > > is > > > > > > > >> only > > > > > > > >> > > > allowed > > > > > > > >> > > > >> >> until the max life time of token. Alternatively > we > > > > could > > > > > > > also > > > > > > > >> make > > > > > > > >> > > > that > > > > > > > >> > > > >> >> an > > > > > > > >> > > > >> >> optional param and the server side default can > > serve > > > > as > > > > > > the > > > > > > > >> upper > > > > > > > >> > > > bound. > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> To your second point it will be done exactly the > > > same > > > > > way > > > > > > we > > > > > > > >> would > > > > > > > >> > > > >> >> support > > > > > > > >> > > > >> >> multiple keytabs. The calling client will have > to > > > put > > > > > the > > > > > > > >> tokens it > > > > > > > >> > > > >> wants > > > > > > > >> > > > >> >> to use in the subject instance and call > > > > produce/consume > > > > > > > inside > > > > > > > >> > > > >> >> subject.doas. Each caller will have to keep > track > > of > > > > its > > > > > > own > > > > > > > >> > > > subject. I > > > > > > > >> > > > >> >> will have to look at the code to see if we > support > > > > this > > > > > > > >> feature > > > > > > > >> > > right > > > > > > > >> > > > >> now > > > > > > > >> > > > >> >> but my understanding is delegation token > shouldn't > > > > need > > > > > > any > > > > > > > >> special > > > > > > > >> > > > >> >> treatment as its just another type of Credential > > in > > > > the > > > > > > > >> subject. > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> I would also like to know what is your opinion > > about > > > > > > > infinite > > > > > > > >> > > renewal > > > > > > > >> > > > >> (my > > > > > > > >> > > > >> >> recommendation is to not support this), tokens > > > > renewing > > > > > > them > > > > > > > >> > > self(my > > > > > > > >> > > > >> >> recommendation is to not support this) and most > > > > > > importantly > > > > > > > >> your > > > > > > > >> > > > choice > > > > > > > >> > > > >> >> between the alternatives listed on this thread > > > > > > > >> > > > >> >> < > > > > > > > >> > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism > > > > > > > >> > > > >> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2 minus > > > > > controller > > > > > > > >> > > > distributing > > > > > > > >> > > > >> >> secret). Thanks again for reviewing. > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> Thanks > > > > > > > >> > > > >> >> Parth > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen Shapira < > > > > > > > >> g...@confluent.io> > > > > > > > >> > > > wrote: > > > > > > > >> > > > >> >> > > > > > > > >> > > > >> >> > Harsha, > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I didn't see > > > your > > > > > > design > > > > > > > >> yet, > > > > > > > >> > > > but in > > > > > > > >> > > > >> >> > our proxy, we have a set of producers, which > > will > > > > > serve > > > > > > > >> multiple > > > > > > > >> > > > users > > > > > > > >> > > > >> >> > going through the proxy. Since these users > will > > > have > > > > > > > >> different > > > > > > > >> > > > >> >> > privileges, they'll need to authenticate > > > separately, > > > > > and > > > > > > > >> can't > > > > > > > >> > > > share a > > > > > > > >> > > > >> >> > token. > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> >> > Am I missing anything? > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> >> > Gwen > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha < > > > > > ka...@harsha.io > > > > > > > > > > > > > > >> wrote: > > > > > > > >> > > > >> >> > > Gwen, > > > > > > > >> > > > >> >> > > On your second point. Can you > > > describe > > > > a > > > > > > > >> usecase > > > > > > > >> > > where > > > > > > > >> > > > >> >> > > mutliple clients ended up > sharing a > > > > > > producer > > > > > > > >> and > > > > > > > >> > > even > > > > > > > >> > > > if > > > > > > > >> > > > >> they > > > > > > > >> > > > >> >> > > do why can't they not use single > > > token > > > > > that > > > > > > > >> producer > > > > > > > >> > > > >> >> > > captures. Why would we need > > multiple > > > > > > clients > > > > > > > >> with > > > > > > > >> > > > >> different > > > > > > > >> > > > >> >> > > tokens sharing a single instance > of > > > > > > producer. > > > > > > > >> Also > > > > > > > >> > > in > > > > > > > >> > > > >> this > > > > > > > >> > > > >> >> > > case other clients have access > all > > > the > > > > > > tokens > > > > > > > >> no? > > > > > > > >> > > > >> >> > > > > > > > > > >> > > > >> >> > > Thanks, > > > > > > > >> > > > >> >> > > Harsha > > > > > > > >> > > > >> >> > > > > > > > > > >> > > > >> >> > > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, Gwen > Shapira > > > > > wrote: > > > > > > > >> > > > >> >> > >> Sorry for the delay: > > > > > > > >> > > > >> >> > >> > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in the > wiki: > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for delegation > > > tokens? > > > > > > > >> Renewal? How > > > > > > > >> > > > do we > > > > > > > >> > > > >> >> > >> revoke them? > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation tokens for > > > > "do-as" > > > > > > > (say, > > > > > > > >> > > submit > > > > > > > >> > > > >> Storm > > > > > > > >> > > > >> >> > >> job as my user), we will need a producer > for > > > > every > > > > > > job > > > > > > > >> (we > > > > > > > >> > > can't > > > > > > > >> > > > >> share > > > > > > > >> > > > >> >> > >> them between multiple jobs running on same > > > node), > > > > > > since > > > > > > > >> we > > > > > > > >> > > only > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is there a > plan > > > to > > > > > > change > > > > > > > >> this > > > > > > > >> > > for > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow > multiple > > > > users > > > > > > > with > > > > > > > >> > > > different > > > > > > > >> > > > >> >> > >> tokens to share a client? > > > > > > > >> > > > >> >> > >> > > > > > > > >> > > > >> >> > >> Gwen > > > > > > > >> > > > >> >> > >> > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, parth > > > brahmbhatt > > > > > > > >> > > > >> >> > >> <brahmbhatt.pa...@gmail.com> wrote: > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can other > > > > > committers > > > > > > > >> review? > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > >> > Thanks > > > > > > > >> > > > >> >> > >> > Parth > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM, Harsha < > > > > > > > >> ka...@harsha.io> > > > > > > > >> > > > wrote: > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> Parth, > > > > > > > >> > > > >> >> > >> >> Overall current design looks > > good > > > to > > > > > > me. I > > > > > > > >> am +1 > > > > > > > >> > > on > > > > > > > >> > > > >> the > > > > > > > >> > > > >> >> > KIP. > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as well. > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > >> >> -Harsha > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 AM, parth > > > > > > brahmbhatt > > > > > > > >> wrote: > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of infinite > > lifetime > > > > > but I > > > > > > > >> see the > > > > > > > >> > > > >> Streaming > > > > > > > >> > > > >> >> > use > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use case I > was > > > > hoping > > > > > > > >> there will > > > > > > > >> > > > be > > > > > > > >> > > > >> some > > > > > > > >> > > > >> >> > notion > > > > > > > >> > > > >> >> > >> >> > of > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new > > delegation > > > > > tokens > > > > > > > at > > > > > > > >> fixed > > > > > > > >> > > > >> interval > > > > > > > >> > > > >> >> > and > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that is not > > the > > > > case > > > > > > for > > > > > > > >> we can > > > > > > > >> > > > >> discuss > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing them self > and > > > the > > > > > > > >> security > > > > > > > >> > > > >> implications > > > > > > > >> > > > >> >> > of the > > > > > > > >> > > > >> >> > >> >> > same. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch tokens > > > from > > > > > > > >> zookeeper, > > > > > > > >> > > > >> overall I > > > > > > > >> > > > >> >> > think > > > > > > > >> > > > >> >> > >> >> > its > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely on our > > > metadata > > > > > > store > > > > > > > >> and I > > > > > > > >> > > > >> think we > > > > > > > >> > > > >> >> > are > > > > > > > >> > > > >> >> > >> >> > moving in that direction with all the > > > KIP-4 > > > > > > > >> improvements. > > > > > > > >> > > > I > > > > > > > >> > > > >> chose > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the client > > will > > > > > still > > > > > > > >> just talk > > > > > > > >> > > > to > > > > > > > >> > > > >> >> > broker , its > > > > > > > >> > > > >> >> > >> >> > the brokers that will use zookeeper > > which > > > we > > > > > > > >> already do > > > > > > > >> > > > for a > > > > > > > >> > > > >> lot > > > > > > > >> > > > >> >> > of > > > > > > > >> > > > >> >> > >> >> > other > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development + and > the > > > > > ability > > > > > > > so > > > > > > > >> > > tokens > > > > > > > >> > > > >> will > > > > > > > >> > > > >> >> > survive > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster > failure. > > > if a > > > > > > > >> majority > > > > > > > >> > > > agrees > > > > > > > >> > > > >> the > > > > > > > >> > > > >> >> > added > > > > > > > >> > > > >> >> > >> >> > complexity to have controller > forwarding > > > > keys > > > > > to > > > > > > > all > > > > > > > >> > > > broker is > > > > > > > >> > > > >> >> > justified > > > > > > > >> > > > >> >> > >> >> > as > > > > > > > >> > > > >> >> > >> >> > it provides tighter security , I am > fine > > > > with > > > > > > that > > > > > > > >> option > > > > > > > >> > > > too. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not support SSL > we > > > can > > > > > not > > > > > > > >> store > > > > > > > >> > > > master > > > > > > > >> > > > >> keys > > > > > > > >> > > > >> >> > in > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will be > exposed > > > on > > > > > > wire. > > > > > > > To > > > > > > > >> > > > support > > > > > > > >> > > > >> >> > rotation > > > > > > > >> > > > >> >> > >> >> > without affecting current clients is > > > > > something I > > > > > > > >> need to > > > > > > > >> > > > put > > > > > > > >> > > > >> more > > > > > > > >> > > > >> >> > thought > > > > > > > >> > > > >> >> > >> >> > in. My current proposal assumes the > > > rotation > > > > > > will > > > > > > > >> > > > invalidate > > > > > > > >> > > > >> all > > > > > > > >> > > > >> >> > current > > > > > > > >> > > > >> >> > >> >> > tokens. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also review > and > > > post > > > > > > their > > > > > > > >> > > comments > > > > > > > >> > > > >> so we > > > > > > > >> > > > >> >> > can > > > > > > > >> > > > >> >> > >> >> > make > > > > > > > >> > > > >> >> > >> >> > progress on this KIP. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks > > > > > > > >> > > > >> >> > >> >> > Parth > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 AM, > Ashish > > > > Singh > > > > > < > > > > > > > >> > > > >> asi...@cloudera.com > > > > > > > >> > > > >> >> > > > > > > > > > >> > > > >> >> > >> >> > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at 11:26 AM, > > > Harsha < > > > > > > > >> > > > ka...@harsha.io> > > > > > > > >> > > > >> >> > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion > threads > > on > > > > > this > > > > > > > KIP. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from Jitendra > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large number of > > > clients > > > > > that > > > > > > > are > > > > > > > >> > > > running > > > > > > > >> > > > >> all > > > > > > > >> > > > >> >> > over the > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate with > Kafka > > > > > > brokers, > > > > > > > >> is very > > > > > > > >> > > > >> similar > > > > > > > >> > > > >> >> > to the > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large number of > > > tasks > > > > > > > running > > > > > > > >> > > across > > > > > > > >> > > > >> the > > > > > > > >> > > > >> >> > cluster > > > > > > > >> > > > >> >> > >> >> that > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs > > Namenode. > > > > > > > >> Therefore, the > > > > > > > >> > > > >> >> > delegation token > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a good fit > > for > > > > > this > > > > > > > use > > > > > > > >> case > > > > > > > >> > > > as we > > > > > > > >> > > > >> >> > have seen > > > > > > > >> > > > >> >> > >> >> it > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in HDFS and > > > YARN. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is very much > > > > inline > > > > > > with > > > > > > > >> Hadoop > > > > > > > >> > > > >> >> > approach. A few > > > > > > > >> > > > >> >> > >> >> > > > comments: > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to allow > > > > infinite > > > > > > > >> renewable > > > > > > > >> > > > >> lifetime > > > > > > > >> > > > >> >> > for a > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a token to a > > max > > > > > life > > > > > > > time > > > > > > > >> > > > (default > > > > > > > >> > > > >> 7 > > > > > > > >> > > > >> >> > days). A > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is believed > to > > > > > > increase > > > > > > > >> with > > > > > > > >> > > > time. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite > lifetime > > > > might > > > > > > not > > > > > > > >> be the > > > > > > > >> > > > best > > > > > > > >> > > > >> idea. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the tokens are > > > stored > > > > > in > > > > > > > >> zookeeper > > > > > > > >> > > > as > > > > > > > >> > > > >> well, > > > > > > > >> > > > >> >> > and > > > > > > > >> > > > >> >> > >> >> can > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is clever > as > > it > > > > can > > > > > > > allow > > > > > > > >> > > > >> replacing the > > > > > > > >> > > > >> >> > tokens > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max life > time, > > > and > > > > > > > clients > > > > > > > >> can > > > > > > > >> > > > >> download > > > > > > > >> > > > >> >> > new > > > > > > > >> > > > >> >> > >> >> tokens > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It shouldn't be a > > big > > > > load > > > > > > on > > > > > > > >> > > zookeeper > > > > > > > >> > > > >> as a > > > > > > > >> > > > >> >> > client > > > > > > > >> > > > >> >> > >> >> will > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token once in > > > several > > > > > > days. > > > > > > > >> In this > > > > > > > >> > > > >> approach > > > > > > > >> > > > >> >> > you > > > > > > > >> > > > >> >> > >> >> don't > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on the > token > > > even > > > > > for > > > > > > > >> long > > > > > > > >> > > > running > > > > > > > >> > > > >> >> > clients. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are > generated > > > > using > > > > > a > > > > > > > >> master > > > > > > > >> > > key. > > > > > > > >> > > > >> The > > > > > > > >> > > > >> >> > master > > > > > > > >> > > > >> >> > >> >> key > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically > changed. > > > In > > > > > > > Hadoop, > > > > > > > >> the > > > > > > > >> > > > >> default > > > > > > > >> > > > >> >> > renewal > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.? > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require brokers > > > > maintaining > > > > > a > > > > > > > >> list of X > > > > > > > >> > > > most > > > > > > > >> > > > >> >> > recent > > > > > > > >> > > > >> >> > >> >> master > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to be > > > persisted > > > > > > > >> somewhere, as > > > > > > > >> > > > if a > > > > > > > >> > > > >> >> > broker > > > > > > > >> > > > >> >> > >> >> goes > > > > > > > >> > > > >> >> > >> >> > > down it will have to get that list > > again > > > > and > > > > > > > >> storing > > > > > > > >> > > > master > > > > > > > >> > > > >> keys > > > > > > > >> > > > >> >> > on ZK > > > > > > > >> > > > >> >> > >> >> is > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However, if a > > broker > > > > goes > > > > > > > down > > > > > > > >> then > > > > > > > >> > > we > > > > > > > >> > > > >> have > > > > > > > >> > > > >> >> > much > > > > > > > >> > > > >> >> > >> >> bigger > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and client can > > always > > > > > > > >> > > re-authenticate > > > > > > > >> > > > is > > > > > > > >> > > > >> such > > > > > > > >> > > > >> >> > >> >> events. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a look at > other > > > > > > > >> alternatives > > > > > > > >> > > this > > > > > > > >> > > > >> list has > > > > > > > >> > > > >> >> > >> >> > > suggested? > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough proposal, > > great > > > > > work!" > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at 10:28 PM, > > Gwen > > > > > > Shapira > > > > > > > >> wrote: > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me. Thanks! > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at 9:25 PM, > > > > Harsha < > > > > > > > >> > > > ka...@harsha.io > > > > > > > >> > > > >> > > > > > > > > >> > > > >> >> > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any release > > > vehicle > > > > > but > > > > > > > >> still the > > > > > > > >> > > > >> work can > > > > > > > >> > > > >> >> > move > > > > > > > >> > > > >> >> > >> >> > > > forward. > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is interested in the > > KIP > > > > > > please > > > > > > > >> do the > > > > > > > >> > > > >> review and > > > > > > > >> > > > >> >> > >> >> provide > > > > > > > >> > > > >> >> > >> >> > > the > > > > > > > >> > > > >> >> > >> >> > > > > > comments. > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at 04:59 > > PM, > > > > > Ismael > > > > > > > >> Juma > > > > > > > >> > > > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would be good > > to > > > > have > > > > > > > more > > > > > > > >> time > > > > > > > >> > > to > > > > > > > >> > > > >> review > > > > > > > >> > > > >> >> > and > > > > > > > >> > > > >> >> > >> >> > > discuss > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at 12:55 > > AM, > > > > Gwen > > > > > > > >> Shapira < > > > > > > > >> > > > >> >> > >> >> g...@confluent.io> > > > > > > > >> > > > >> >> > >> >> > > > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 depends on > > KIP-43, > > > > > which > > > > > > > is > > > > > > > >> > > > already a > > > > > > > >> > > > >> bit > > > > > > > >> > > > >> >> > of a > > > > > > > >> > > > >> >> > >> >> risk > > > > > > > >> > > > >> >> > >> >> > > for > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release - any > chance > > > we > > > > > can > > > > > > > >> delay > > > > > > > >> > > > >> delegation > > > > > > > >> > > > >> >> > tokens > > > > > > > >> > > > >> >> > >> >> to > > > > > > > >> > > > >> >> > >> >> > > > Kafka > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1? > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community working > > on a > > > > > > release > > > > > > > >> every > > > > > > > >> > > 3 > > > > > > > >> > > > >> month, > > > > > > > >> > > > >> >> > this > > > > > > > >> > > > >> >> > >> >> is not > > > > > > > >> > > > >> >> > >> >> > > > a huge > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, 2016 at > 5:11 > > > PM, > > > > > > Ashish > > > > > > > >> Singh > > > > > > > >> > > < > > > > > > > >> > > > >> >> > >> >> > > asi...@cloudera.com> > > > > > > > >> > > > >> >> > >> >> > > > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for the > > awesome > > > > > write > > > > > > > up. > > > > > > > >> > > > Following > > > > > > > >> > > > >> our > > > > > > > >> > > > >> >> > >> >> discussion > > > > > > > >> > > > >> >> > >> >> > > > from the > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it will be > > > easier > > > > > to > > > > > > > >> compare > > > > > > > >> > > > >> various > > > > > > > >> > > > >> >> > >> >> alternatives > > > > > > > >> > > > >> >> > >> >> > > > if they > > > > > > > >> > > > >> >> > >> >> > > > > >> > are > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together. I am > > stating > > > > > > below a > > > > > > > >> few > > > > > > > >> > > > >> >> > alternatives along > > > > > > > >> > > > >> >> > >> >> > > with > > > > > > > >> > > > >> >> > >> >> > > > a the > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal) Store > > > > > Delegation > > > > > > > >> Token, > > > > > > > >> > > DT, > > > > > > > >> > > > >> on ZK. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > authenticates > > > > with a > > > > > > > >> broker. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is > > > > > > authenticated, > > > > > > > >> it > > > > > > > >> > > will > > > > > > > >> > > > >> make a > > > > > > > >> > > > >> >> > broker > > > > > > > >> > > > >> >> > >> >> side > > > > > > > >> > > > >> >> > >> >> > > > call to > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation > > token. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker > generates > > a > > > > > shared > > > > > > > >> secret > > > > > > > >> > > > based > > > > > > > >> > > > >> on > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret shared > > > > between > > > > > > all > > > > > > > >> > > brokers, > > > > > > > >> > > > >> >> > randomly > > > > > > > >> > > > >> >> > >> >> > > generated > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId). > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker stores this > > > token > > > > in > > > > > > its > > > > > > > >> in > > > > > > > >> > > > memory > > > > > > > >> > > > >> cache. > > > > > > > >> > > > >> >> > >> >> Broker > > > > > > > >> > > > >> >> > >> >> > > > also stores > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the DelegationToken > > > without > > > > > the > > > > > > > >> hmac in > > > > > > > >> > > the > > > > > > > >> > > > >> >> > zookeeper. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers will > > have a > > > > > cache > > > > > > > >> backed > > > > > > > >> > > by > > > > > > > >> > > > >> >> > zookeeper so > > > > > > > >> > > > >> >> > >> >> they > > > > > > > >> > > > >> >> > >> >> > > > will all > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified whenever > a > > > new > > > > > > token > > > > > > > is > > > > > > > >> > > > >> generated and > > > > > > > >> > > > >> >> > they > > > > > > > >> > > > >> >> > >> >> will > > > > > > > >> > > > >> >> > >> >> > > > update > > > > > > > >> > > > >> >> > >> >> > > > > >> > their > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache whenever > > token > > > > > state > > > > > > > >> changes. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker returns the > > > token > > > > to > > > > > > > >> Client. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable race > > > condition, > > > > > > client > > > > > > > >> tries > > > > > > > >> > > to > > > > > > > >> > > > >> >> > authenticate > > > > > > > >> > > > >> >> > >> >> with > > > > > > > >> > > > >> >> > >> >> > > > a broker > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to be > > updated > > > > with > > > > > > the > > > > > > > >> newly > > > > > > > >> > > > >> generated > > > > > > > >> > > > >> >> > DT. > > > > > > > >> > > > >> >> > >> >> This > > > > > > > >> > > > >> >> > >> >> > > can > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with making > > > dtRequest > > > > > > block > > > > > > > >> until > > > > > > > >> > > all > > > > > > > >> > > > >> >> > brokers have > > > > > > > >> > > > >> >> > >> >> > > > updated > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk barrier or > > > similar > > > > > > > >> mechanism > > > > > > > >> > > can > > > > > > > >> > > > be > > > > > > > >> > > > >> used. > > > > > > > >> > > > >> >> > >> >> However, > > > > > > > >> > > > >> >> > >> >> > > > all such > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will > increase > > > > > > > complexity. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a static > secret > > > key > > > > > > from > > > > > > > >> config > > > > > > > >> > > > >> file. Will > > > > > > > >> > > > >> >> > >> >> require > > > > > > > >> > > > >> >> > >> >> > > yet > > > > > > > >> > > > >> >> > >> >> > > > > >> > another > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and uses a > static > > > > > secret > > > > > > > >> key. It > > > > > > > >> > > is > > > > > > > >> > > > >> advised > > > > > > > >> > > > >> >> > to > > > > > > > >> > > > >> >> > >> >> rotate > > > > > > > >> > > > >> >> > >> >> > > > secret > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. This can > > be > > > > > > avoided > > > > > > > >> with > > > > > > > >> > > > >> controller > > > > > > > >> > > > >> >> > >> >> generating > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to brokers > > > > > periodically. > > > > > > > >> However, > > > > > > > >> > > > >> this will > > > > > > > >> > > > >> >> > >> >> require > > > > > > > >> > > > >> >> > >> >> > > > brokers to > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain certain > counts > > of > > > > > > > >> secretKeys. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1) Have > > > controller > > > > > > > >> generate > > > > > > > >> > > > >> delegation > > > > > > > >> > > > >> >> > token. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > authenticates > > > > with a > > > > > > > >> broker. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is > > > > > > authenticated, > > > > > > > >> it > > > > > > > >> > > will > > > > > > > >> > > > >> make a > > > > > > > >> > > > >> >> > broker > > > > > > > >> > > > >> >> > >> >> side > > > > > > > >> > > > >> >> > >> >> > > > call to > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation > > token. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker forwards the > > > > request > > > > > > to > > > > > > > >> > > > controller. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller > generates > > a > > > DT > > > > > and > > > > > > > >> > > broadcasts > > > > > > > >> > > > >> to all > > > > > > > >> > > > >> >> > >> >> brokers. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker stores this > > > token > > > > in > > > > > > its > > > > > > > >> memory > > > > > > > >> > > > >> cache. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller responds > > to > > > > > > broker’s > > > > > > > >> DT > > > > > > > >> > > req. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker returns the > > > token > > > > to > > > > > > > >> Client. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will have to add > > new > > > > > APIs > > > > > > to > > > > > > > >> > > support > > > > > > > >> > > > >> >> > controller > > > > > > > >> > > > >> >> > >> >> pushing > > > > > > > >> > > > >> >> > >> >> > > > tokens > > > > > > > >> > > > >> >> > >> >> > > > > >> > to > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top of the > > > > minimal > > > > > > APIs > > > > > > > >> that > > > > > > > >> > > are > > > > > > > >> > > > >> >> > currently > > > > > > > >> > > > >> >> > >> >> > > proposed. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will also have > to > > > add > > > > > APIs > > > > > > > to > > > > > > > >> > > support > > > > > > > >> > > > >> the > > > > > > > >> > > > >> >> > >> >> bootstrapping > > > > > > > >> > > > >> >> > >> >> > > > case, > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new broker > comes > > up > > > > it > > > > > > will > > > > > > > >> have > > > > > > > >> > > to > > > > > > > >> > > > >> get all > > > > > > > >> > > > >> >> > >> >> delegation > > > > > > > >> > > > >> >> > >> >> > > > tokens > > > > > > > >> > > > >> >> > >> >> > > > > >> > from > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In catastrophic > > > failures > > > > > > where > > > > > > > >> all > > > > > > > >> > > > brokers > > > > > > > >> > > > >> go > > > > > > > >> > > > >> >> > down, > > > > > > > >> > > > >> >> > >> >> the > > > > > > > >> > > > >> >> > >> >> > > > tokens will > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if > servers > > > are > > > > > > > >> restarted as > > > > > > > >> > > > >> tokens > > > > > > > >> > > > >> >> > are not > > > > > > > >> > > > >> >> > >> >> > > > persisted > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens, then > > > there > > > > > are > > > > > > > more > > > > > > > >> > > > important > > > > > > > >> > > > >> >> > things to > > > > > > > >> > > > >> >> > >> >> > > worry > > > > > > > >> > > > >> >> > >> >> > > > about > > > > > > > >> > > > >> >> > >> >> > > > > >> > and > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better to > > > > > > > >> re-authenticate. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2) Do not > > > > distribute > > > > > > DT > > > > > > > to > > > > > > > >> > > other > > > > > > > >> > > > >> brokers > > > > > > > >> > > > >> >> > at > > > > > > > >> > > > >> >> > >> >> all. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > authenticates > > > > with a > > > > > > > >> broker. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is > > > > > > authenticated, > > > > > > > >> it > > > > > > > >> > > will > > > > > > > >> > > > >> make a > > > > > > > >> > > > >> >> > broker > > > > > > > >> > > > >> >> > >> >> side > > > > > > > >> > > > >> >> > >> >> > > > call to > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation > > token. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker > generates > > DT > > > > of > > > > > > > form, > > > > > > > >> > > [hmac + > > > > > > > >> > > > >> (owner, > > > > > > > >> > > > >> >> > >> >> renewer, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime, id, hmac, > > > > > > > >> expirationTime)] > > > > > > > >> > > and > > > > > > > >> > > > >> passes > > > > > > > >> > > > >> >> > back > > > > > > > >> > > > >> >> > >> >> this > > > > > > > >> > > > >> >> > >> >> > > > DT to > > > > > > > >> > > > >> >> > >> >> > > > > >> > client. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is generated via > > > > > > > >> {HMAC-SHA256(owner, > > > > > > > >> > > > >> renewer, > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id, > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) using > > > > > > SecretKey}. > > > > > > > >> Note > > > > > > > >> > > that > > > > > > > >> > > > >> all > > > > > > > >> > > > >> >> > brokers > > > > > > > >> > > > >> >> > >> >> have > > > > > > > >> > > > >> >> > >> >> > > > this > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then goes to > > any > > > > > > broker > > > > > > > >> and to > > > > > > > >> > > > >> >> > authenticate > > > > > > > >> > > > >> >> > >> >> sends > > > > > > > >> > > > >> >> > >> >> > > > the DT. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker recalculates > hmac > > > > using > > > > > > > >> (owner, > > > > > > > >> > > > >> renewer, > > > > > > > >> > > > >> >> > >> >> maxLifeTime, > > > > > > > >> > > > >> >> > >> >> > > > id, hmac, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) info > > from > > > DT > > > > > and > > > > > > > its > > > > > > > >> > > > >> SecretKey. If > > > > > > > >> > > > >> >> > it > > > > > > > >> > > > >> >> > >> >> matches > > > > > > > >> > > > >> >> > >> >> > > > with > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is > > > authenticated. > > > > > > Yes, > > > > > > > >> it will > > > > > > > >> > > > do > > > > > > > >> > > > >> other > > > > > > > >> > > > >> >> > >> >> obvious > > > > > > > >> > > > >> >> > >> >> > > > checks of > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp expiry and > > such. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret key will > > be > > > > > > > generated > > > > > > > >> by > > > > > > > >> > > > >> controller > > > > > > > >> > > > >> >> > and > > > > > > > >> > > > >> >> > >> >> passed > > > > > > > >> > > > >> >> > >> >> > > to > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to delete a DT? > > > Yes, > > > > > that > > > > > > > is > > > > > > > >> a > > > > > > > >> > > > downside > > > > > > > >> > > > >> >> > here. > > > > > > > >> > > > >> >> > >> >> However, > > > > > > > >> > > > >> >> > >> >> > > > this can > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with > brokers > > > > > > > maintaining > > > > > > > >> a > > > > > > > >> > > > >> blacklist of > > > > > > > >> > > > >> >> > DTs, > > > > > > > >> > > > >> >> > >> >> DTs > > > > > > > >> > > > >> >> > >> >> > > > from this > > > > > > > >> > > > >> >> > >> >> > > > > >> > list > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed after > > > expiry. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In catastrophic > > > failures > > > > > > where > > > > > > > >> all > > > > > > > >> > > > brokers > > > > > > > >> > > > >> go > > > > > > > >> > > > >> >> > down, > > > > > > > >> > > > >> >> > >> >> the > > > > > > > >> > > > >> >> > >> >> > > > tokens will > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if > servers > > > are > > > > > > > >> restarted as > > > > > > > >> > > > >> tokens > > > > > > > >> > > > >> >> > are not > > > > > > > >> > > > >> >> > >> >> > > > persisted > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens, then > > > there > > > > > are > > > > > > > more > > > > > > > >> > > > important > > > > > > > >> > > > >> >> > things to > > > > > > > >> > > > >> >> > >> >> > > worry > > > > > > > >> > > > >> >> > >> >> > > > about > > > > > > > >> > > > >> >> > >> >> > > > > >> > and > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better to > > > > > > > >> re-authenticate. > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, 2016 at > > 1:58 > > > > PM, > > > > > > > Parth > > > > > > > >> > > > >> Brahmbhatt < > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > pbrahmbh...@hortonworks.com> > > > > > > wrote: > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi, > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed KIP-48 so > we > > > can > > > > > > offer > > > > > > > >> hadoop > > > > > > > >> > > > like > > > > > > > >> > > > >> >> > delegation > > > > > > > >> > > > >> >> > >> >> > > > tokens in > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can review > the > > > > design > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka > > > > > > > >> > > > >> >> > >> >> > > > > >> > . > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends on > KIP-43 > > > and > > > > > we > > > > > > > >> have also > > > > > > > >> > > > >> >> > discussed an > > > > > > > >> > > > >> >> > >> >> > > > alternative to > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design here< > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800 > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >. > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > -- > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards, > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > -- > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > Regards, > > > > > > > >> > > > >> >> > >> >> > > Ashish > > > > > > > >> > > > >> >> > >> >> > > > > > > > > > >> > > > >> >> > >> >> > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Regards, > > > > > > > > Rajini > > > > > > > > > > > > > > > -- > > Regards, > > > > Rajini > > >
