Both the KIP and the PR have been updated to a cut-down version as discussed in the KIP meeting today.
Any feedback is appreciated. On Tue, Mar 15, 2016 at 7:39 PM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > Following on from the discussions in the KIP meeting today, the suggestion > is to implement a cut-down version of KIP-43 for 0.10.0.0 with a follow-on > KIP after the release to address support for custom mechanisms. > > Changes to be removed from KIP-43: > > 1. Remove the configuration for CallbackHandler. The callback handler > implementation in Kafka will support Kerberos, PLAIN and Digest-MD5. It > will not support custom or more complex mechanisms which require additional > callbacks. > 2. Remove the configuration for Login. The Login implementation in > Kafka will support Kerberos and any other mechanism (PLAIN, Digest-MD5 etc) > that doesn't require functionality like token refresh. > > Changes included in KIP-43: > > 1. Configurable mechanism > 2. Support for multiple mechanisms in the broker > 3. Implementation of SASL/PLAIN > > If there are no objections to this, I can update the KIP and the PR by > tomorrow. And move the support for custom mechanisms into another KIP and > PR for review after the release of 0.10.0.0. > > > On Mon, Mar 14, 2016 at 7:48 AM, Rajini Sivaram < > rajinisiva...@googlemail.com> wrote: > >> Harsha, >> >> You are right, we don't expect to override callback handler or login for >> Digest-MD5. >> >> Pluggable CallbackHandler and Login modules enable custom SASL mechanisms >> to be implemented without modifying Kafka. For instance, it would enable >> KIP-44 ( >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-44+-+Allow+Kafka+to+have+a+customized+security+protocol) >> to be implemented without making the whole security protocol pluggable. Tao >> Xiao has already confirmed earlier in this discussion thread that the >> proposed callback handler and login interfaces are suitable for their >> custom authentication. >> >> >> >> On Sun, Mar 13, 2016 at 6:59 PM, Harsha <ka...@harsha.io> wrote: >> >>> Agree with Gwen here. I feel like these additional pluggable Login >>> Modules are making this KIP complex. Since the main goal of the KIP is >>> to enable additional mechanism , can we limit the scope to that and If >>> we feel necessary for pluggable Login and callback handler classes we >>> can address in another JIRA. >>> >>> Adding digest-md5 ,password callbacks can be done to existing >>> callbackhandler without expose it as pluggable class. It would be >>> useful to have broker support multiple mechanisms. I haven't seen >>> anyone using more than this in hadoop . It might be different for Kafka >>> but I personally haven't seen anyone asking for this yet. >>> >>> Thanks, >>> Harsha >>> >>> >>> On Thu, Mar 10, 2016, at 01:44 AM, Rajini Sivaram wrote: >>> > Gwen, >>> > >>> > Just to be clear, the alternative would be: >>> > >>> > *jaas.conf:* >>> > >>> > GssapiKafkaServer { >>> > >>> > com.ibm.security.auth.module.Krb5LoginModule required >>> > credsType=both >>> > useKeytab="file:/kafka/key.tab" >>> > principal="kafka/localh...@example.com <http://example.com/>"; >>> > >>> > }; >>> > >>> > SmartcardKafkaServer { >>> > >>> > example.SmartcardLoginModule required >>> > >>> > cardNumber=123; >>> > >>> > }; >>> > >>> > >>> > *KafkaConfig* >>> > >>> > >>> > >>> > - login.context.map={"GSSAPI="GssapiKafkaServer", >>> > "SMARTCARD"=SmartcardKafkaServer} >>> > - login.class.map={"GSSAPI=GssapiLogin.class, >>> > "SMARTCARD"=SmartcardLogin.class} >>> > - >>> callback.handler.class.map={"GSSAPI"=GssapiCallbackHandler.class, >>> > "SMARTCARD"=SmartcardCallbackHandler.class} >>> > >>> > *Client Config * >>> > Same as the server, but with only one entry allowed in each map and >>> > jaas.conf >>> > >>> > >>> > >>> > This is a different model from the Java standard for supporting >>> multiple >>> > logins. As a developer, I am inclined to stick with approaches that are >>> > widely in use like JSSE. But this alternative can be made to work if >>> the >>> > Kafka community feels it is more appropriate for Kafka. If you know of >>> > other systems which use this approach, that would be helpful. >>> > >>> > >>> > >>> > On Thu, Mar 10, 2016 at 2:07 AM, Gwen Shapira <g...@confluent.io> >>> wrote: >>> > >>> > > What I'm hearing is that: >>> > > >>> > > 1. In order to support authentication mechanisms that were not >>> written >>> > > specifically with Kafka in mind, someone will need to write the >>> > > integration between the mechanism and Kafka. This may include Login >>> > > and CallbackHandler classes. This can be the mechanism vendor, the >>> > > user or a 3rd party vendor. >>> > > 2. If someone wrote the code to support a mechanism in Kafka, and a >>> > > user will want to use more than one mechanism, they will still need >>> to >>> > > write a wrapper. >>> > > 3. In reality, #2 will not be necessary ("edge-case") because Kafka >>> > > will actually already provide the callback needed (and presumably >>> also >>> > > the code to load the LoginModule provided by Example.com)? >>> > > >>> > > Tradeoff #1 sounds reasonable. >>> > > #2 and #3 do not sound reasonable considering one of the goals of the >>> > > patch is to support multiple mechanisms. I don't think we should >>> force >>> > > our users to write code just to avoid writing it ourselves. >>> > > Configuring security is complex enough as is. >>> > > Furthermore, if we believe that "Smartcard is likely to use standard >>> > > NameCallback and PasswordCallback already implemented in Kafka" - why >>> > > do we even provide configuration for Login and CallbackHandler >>> > > classes? Either we support multiple mechanisms written by different >>> > > vendors, or we don't. >>> > > >>> > > Gwen >>> > > >>> > > >>> > > On Wed, Mar 9, 2016 at 12:32 AM, Rajini Sivaram >>> > > <rajinisiva...@googlemail.com> wrote: >>> > > > I am not saying that the developer at Example Inc. would develop a >>> Login >>> > > > implementation that combines Smartcard and Kerberos because >>> Retailer uses >>> > > > both. I am saying that Example Inc develops the LoginModule >>> (similar to >>> > > JVM >>> > > > security providers developing Kerberos modules). But there is no >>> standard >>> > > > interface for Login to allow ticket refresh. So, it is very >>> unlikely that >>> > > > Example Inc would develop a Login implementation that works with an >>> > > Apache >>> > > > Kafka defined interface ( Kafka developers wrote this code for >>> Kerberos). >>> > > > For a custom integration, the user (i.e. Retailer) would be >>> expected to >>> > > > develop this code if required. >>> > > > >>> > > > You could imagine that Smartcard is a commonly used mechanism and >>> a 3rd >>> > > > party develops code for integrating Smartcard with Kafka and makes >>> the >>> > > > integration code (Login and CallbackHandler implementation) widely >>> > > > available, If Retailer wants to use clients or a broker with just >>> > > Smartcard >>> > > > enabled in their broker, they configure Kafka to use the 3rd party >>> code, >>> > > > with no additional code development. But to combine Smartcard and >>> > > Kerberos, >>> > > > Retailer needs to write a few lines of code to incorporate both >>> Smartcard >>> > > > and Kerberos. I believe this is an edge case. >>> > > > >>> > > > Smartcard is likely to use standard NameCallback and >>> PasswordCallback >>> > > > already implemented in Kafka and Kerberos support exists in Kafka. >>> So it >>> > > is >>> > > > very likely that Retailer doesn't need to override Login or >>> > > CallbackHandler >>> > > > in this case. And it would just be a matter of configuring the >>> > > mechanisms. >>> > > > >>> > > > On Wed, Mar 9, 2016 at 12:48 AM, Gwen Shapira <g...@confluent.io> >>> wrote: >>> > > > >>> > > >> "Since smart card logins are not built into Kafka (or the JDK), >>> you >>> > > need a >>> > > >> developer to build the login module. So the developer implements >>> > > >> example.SmartcardLoginModule. In addition, the developer may also >>> > > implement >>> > > >> callback handlers for the SASL client or server and a login >>> class to >>> > > keep >>> > > >> this login refreshed. The callback handlers and login >>> implementation >>> > > >> support all the mechanisms that the organisation supports - in >>> this case >>> > > >> Kerberos and smart card." >>> > > >> >>> > > >> In this case, the developer works for Example Inc (which develops >>> > > >> SmartCard authentication modules), while I work for Retailer and >>> need >>> > > >> to use his module. >>> > > >> You assume that developer from Example Inc knows about all the >>> > > >> mechanisms that I have enabled in my Kafka cluster and he is >>> capable >>> > > >> of developing a Login class and maybe a callback handler that >>> supports >>> > > >> all of them. I think this is an unreasonable expectation. >>> > > >> >>> > > >> I think that allowing plug in of Login method and single Callback >>> > > >> handler is not a good extensible solution for multiple >>> mechanisms. We >>> > > >> should probably supply a Kafka-based LoginContext and >>> CallbackHandler >>> > > >> that will be able to delegate to implementations of LoginModules >>> and >>> > > >> Callback Handlers based on mechanism out of a list supplied by a >>> user. >>> > > >> I agree it is more complex, but otherwise we are asking of too >>> much >>> > > >> coordination from whoever implements CallbackHandlers and Login >>> > > >> methods. >>> > > >> >>> > > >> Gwen >>> > > >> >>> > > >> On Tue, Mar 8, 2016 at 3:35 PM, Rajini Sivaram >>> > > >> <rajinisiva...@googlemail.com> wrote: >>> > > >> > As an admin, you would configure login modules using standard >>> JAAS >>> > > >> > configuration. So, if you are supporting Kerberos and smart >>> card login >>> > > >> > modules within a single broker, in your jaas.conf you would >>> specify: >>> > > >> > >>> > > >> > KafkaServer { >>> > > >> > >>> > > >> > com.ibm.security.auth.module.Krb5LoginModule required >>> > > >> > credsType=both >>> > > >> > useKeytab="file:/kafka/key.tab" >>> > > >> > principal="kafka/localh...@example.com"; >>> > > >> > >>> > > >> > example.SmartcardLoginModule required >>> > > >> > >>> > > >> > cardNumber=123 >>> > > >> > >>> > > >> > }; >>> > > >> > >>> > > >> > >>> > > >> > Since smart card logins are not built into Kafka (or the JDK), >>> you >>> > > need a >>> > > >> > developer to build the login module. So the developer implements >>> > > >> > example.SmartcardLoginModule. In addition, the developer may >>> also >>> > > >> implement >>> > > >> > callback handlers for the SASL client or server and a login >>> class to >>> > > >> keep >>> > > >> > this login refreshed. The callback handlers and login >>> implementation >>> > > >> > support all the mechanisms that the organisation supports - in >>> this >>> > > case >>> > > >> > Kerberos and smart card. >>> > > >> > >>> > > >> > As an admin, if you are using mechanisms built into Kafka, you >>> simply >>> > > >> > specify a JAAS config with the combination of mechanisms you >>> want and >>> > > you >>> > > >> > are done. If you are adding new mechanisms, it clearly can't be >>> done >>> > > >> > without some additional code to implement that mechanism. The >>> callback >>> > > >> and >>> > > >> > login interfaces are configurable to ensure that (as a >>> developer), you >>> > > >> can >>> > > >> > build custom mechanisms without modifying Kafka. SASL was >>> designed >>> > > >> > precisely to do this. But at the moment, Kafka restricts what >>> type of >>> > > >> > authentication you can perform with SASL. >>> > > >> > >>> > > >> > Back to the questions: >>> > > >> > 1. *How does the proposed Login class interact with >>> LoginContext and * >>> > > >> > *LoginModule.* >>> > > >> > Login class performs *LoginContext.login(). *This performs >>> login for >>> > > all >>> > > >> > the login modules of that context (this is standard Java >>> function). >>> > > Login >>> > > >> > is cached with a reference counter to keep it alive. >>> > > >> > >>> > > >> > 2. *Do we really need a pluggable Login class, when users >>> > > >> **(administrators, >>> > > >> > not developers) are more likely to want to specify >>> LoginModules?* >>> > > >> > Administrators specify *LoginModules. *Developers implement new >>> > > >> mechanisms. >>> > > >> > The pluggable *Login* class enables the developer to implement >>> new >>> > > >> > mechanisms without changing Kafka. If Kafka didn't have a >>> default >>> > > >> Kerberos >>> > > >> > implementation and a user wanted to use Kerberos, pluggability >>> of >>> > > >> *Login* class >>> > > >> > enables the user to include Kerberos without changing Kafka. >>> Not that >>> > > we >>> > > >> > want another Kerberos, but Tao had a requirement to enable a >>> custom >>> > > >> > mechanism with functionality similar to Kerberos (see KIP-44). >>> > > Pluggable >>> > > >> > *Login* supports this use case without replacing the entire >>> > > >> authentication >>> > > >> > mechanism in Kafka. >>> > > >> > >>> > > >> > *3. How will users get to specify multiple LoginModules?* >>> > > >> > Standard Java JAAS configuration - see example above. >>> > > >> > >>> > > >> > >>> > > >> > *4. It sounds like different LoginModules may need >>> > > >> > different CallbackHandlers. Since we only allow a single >>> > > >> > pluggableCallbackHandler class in the KIP-43 configuration, how >>> will >>> > > this >>> > > >> > be handled?* >>> > > >> > One LoginContext, one subject, one callback handler, multiple >>> login >>> > > >> > modules. This is the Java model for login. And it works well >>> because >>> > > >> > callback handlers tend to be fairly standard. Even if the smart >>> card >>> > > SASL >>> > > >> > callbacks needed to do something wildly different and you >>> wanted a >>> > > broker >>> > > >> > that combined this with Kerberos, you would implement a >>> callback that >>> > > >> > looked like the code below. >>> > > >> > >>> > > >> > private CallbackHandler handler; >>> > > >> > >>> > > >> > public void configure(Map<String, ?> configs, Mode mode, >>> > > Subject >>> > > >> > subject, String mechanism) {public void configure(Map<String, ?> >>> > > configs, >>> > > >> > Mode mode, Subject subject, String mechanism) { >>> > > >> > >>> > > >> > switch (mechanism) { >>> > > >> > >>> > > >> > case "GSSAPI" : handler = new >>> GssapiCallbackHandler(); >>> > > >> break >>> > > >> > ; >>> > > >> > >>> > > >> > case "SMARTCARD" : handler = new >>> > > >> SmartcardCallbackHandler(); >>> > > >> > break; >>> > > >> > >>> > > >> > } >>> > > >> > >>> > > >> > } >>> > > >> > >>> > > >> > >>> > > >> > public void handle(Callback[] callbacks) throws >>> IOException, >>> > > >> > UnsupportedCallbackException { >>> > > >> > >>> > > >> > handler.handle(callbacks); >>> > > >> > >>> > > >> > } >>> > > >> > >>> > > >> > But I don't see this as just an admin configuration. This is >>> some code >>> > > >> > developed alongside a new mechanism. Yes, you can add a new >>> > > configuration >>> > > >> > type in Kafka that maps mechanism to callback handler class to >>> avoid >>> > > the >>> > > >> > switch statement. But that doesn't remove the need for new code >>> > > >> altogether >>> > > >> > since you still need to implement SmartcardCallbackHandler. >>> Multiple >>> > > >> > conflicting callback handlers within one broker to support >>> complete >>> > > >> > different behaviour is an edge case. Since the design supports >>> > > >> > full flexibility in terms of functionality and you have to >>> write new >>> > > code >>> > > >> > for new mechanisms anyway, this feels like the simplest >>> solution. >>> > > >> > >>> > > >> > >>> > > >> > >>> > > >> > On Tue, Mar 8, 2016 at 8:23 PM, Gwen Shapira <g...@confluent.io >>> > >>> > > wrote: >>> > > >> > >>> > > >> >> I guess what is unclear to me is: >>> > > >> >> >>> > > >> >> 1. How does the proposed Login class interact with >>> LoginContext and >>> > > >> >> LoginModule. >>> > > >> >> 2. Do we really need a pluggable Login class, when users >>> > > >> >> (administrators, not developers) are more likely to want to >>> specify >>> > > >> >> LoginModules? >>> > > >> >> 3. How will users get to specify multiple LoginModules? >>> > > >> >> 4. It sounds like different LoginModules may need different >>> > > >> >> CallbackHandlers. Since we only allow a single pluggable >>> > > >> >> CallbackHandler class in the KIP-43 configuration, how will >>> this be >>> > > >> >> handled? >>> > > >> >> >>> > > >> >> In other words, suppose I want my Kafka installation to >>> support both >>> > > >> >> Kerberos and SmartCard mechanisms, can you tell me how to >>> configure >>> > > >> >> Kafka? (as admin, not developer)? >>> > > >> >> >>> > > >> >> Gwen >>> > > >> >> >>> > > >> >> >>> > > >> >> >>> > > >> >> On Tue, Mar 8, 2016 at 11:46 AM, Rajini Sivaram >>> > > >> >> <rajinisiva...@googlemail.com> wrote: >>> > > >> >> > Gwen, >>> > > >> >> > >>> > > >> >> > I am not sure I understand your concern. Java login contexts >>> > > support >>> > > >> >> > multiple login modules precisely to support this type of >>> real-world >>> > > >> >> > scenario. >>> > > >> >> > >>> > > >> >> > From Javadoc of *javax.security.auth.login.LoginContext*: >>> > > >> >> > >>> > > >> >> > *In addition to supporting pluggable authentication, this >>> class >>> > > also >>> > > >> >> > supports the notion of stacked authentication. Applications >>> may be >>> > > >> >> > configured to use more than one LoginModule. For example, >>> one could >>> > > >> >> > configure both a Kerberos LoginModule and a smart card >>> LoginModule >>> > > >> under >>> > > >> >> an >>> > > >> >> > application. * >>> > > >> >> > >>> > > >> >> > * A typical caller instantiates a LoginContext with a name >>> and a >>> > > >> >> > CallbackHandler. LoginContext uses the name as the index >>> into a >>> > > >> >> > Configuration to determine which LoginModules should be >>> used, and >>> > > >> which >>> > > >> >> > ones must succeed in order for the overall authentication to >>> > > succeed. >>> > > >> The >>> > > >> >> > CallbackHandler is passed to the underlying LoginModules so >>> they >>> > > may >>> > > >> >> > communicate and interact with users (prompting for a >>> username and >>> > > >> >> password >>> > > >> >> > via a graphical user interface, for example).* >>> > > >> >> > >>> > > >> >> > If it is good enough for Java, surely it must be good enough >>> for >>> > > >> Kafka. >>> > > >> >> We >>> > > >> >> > don't expect users to directly use existing classes when >>> they want >>> > > to >>> > > >> >> > support complex new mechanisms. Login is not a standard >>> interface, >>> > > >> and we >>> > > >> >> > want to make callback handlers configurable. So we do require >>> > > users to >>> > > >> >> wrap >>> > > >> >> > their existing classes in our interface. And the interfaces >>> make it >>> > > >> >> > possible to plugin any number of mechanisms. Am I missing >>> > > something? >>> > > >> >> > >>> > > >> >> > >>> > > >> >> > >>> > > >> >> > On Tue, Mar 8, 2016 at 4:49 PM, Gwen Shapira < >>> g...@confluent.io> >>> > > >> wrote: >>> > > >> >> > >>> > > >> >> >> Yes, I understand that. >>> > > >> >> >> >>> > > >> >> >> However, because the current configuration only allows a >>> single >>> > > >> >> >> callback and a single login module, I need to supply a >>> class that >>> > > >> >> >> supports callbacks or logins for every single mechanism >>> that I >>> > > need >>> > > >> to >>> > > >> >> >> support. I question whether this is applicable in real-world >>> > > scenario >>> > > >> >> >> where a user may need to support mechanisms that were >>> written by >>> > > >> >> >> different vendors and where a single class that implements >>> all the >>> > > >> >> >> necessary modules or callsbacks is very unlikely to exist. >>> > > >> >> >> >>> > > >> >> >> I agree that supporting a list of classes and dynamically >>> using >>> > > the >>> > > >> >> >> correct callbacks and modules based on configuration is >>> complex, >>> > > but >>> > > >> I >>> > > >> >> >> don't think this feature is useful enough without it. >>> > > >> >> >> >>> > > >> >> >> Gwen >>> > > >> >> >> >>> > > >> >> >> On Tue, Mar 8, 2016 at 2:40 AM, Rajini Sivaram >>> > > >> >> >> <rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> > Gwen, >>> > > >> >> >> > >>> > > >> >> >> > Sorry if this is not clear in the KIP. The goal of the >>> KIP is to >>> > > >> >> enable >>> > > >> >> >> new >>> > > >> >> >> > mechanisms to be added without any changes to Apache >>> Kafka. The >>> > > >> >> >> > configuration proposed by the KIP is based on the >>> > > configurability >>> > > >> of >>> > > >> >> the >>> > > >> >> >> > Java SASL API and what it would take to add a new >>> mechanism >>> > > >> similar to >>> > > >> >> >> > Kerberos/PLAIN/Digest-MD5 and the requirements that Tao >>> raised >>> > > for >>> > > >> >> custom >>> > > >> >> >> > authentication. >>> > > >> >> >> > >>> > > >> >> >> > The PR includes a test for Digest-MD5 which is not >>> included in >>> > > >> Kafka, >>> > > >> >> but >>> > > >> >> >> > is tested using the configuration options that allow new >>> > > mechanisms >>> > > >> >> to be >>> > > >> >> >> > added. >>> > > >> >> >> > >>> > > >> >> >> > If you want to add a more complex mechanism which requires >>> > > >> integration >>> > > >> >> >> with >>> > > >> >> >> > some external authentication libraries for login or >>> callback >>> > > >> handlers, >>> > > >> >> >> you >>> > > >> >> >> > can do that by configuring your own callback handler and >>> login >>> > > >> >> >> > implementation. The login class allows you to add >>> additional >>> > > >> threads >>> > > >> >> (for >>> > > >> >> >> > instance for token refresh), while the callback handler >>> extends >>> > > the >>> > > >> >> >> > standard Java CallbackHandler interface, enabling you to >>> > > override >>> > > >> any >>> > > >> >> of >>> > > >> >> >> > the default callbacks in Kafka to provide your own >>> behaviour. If >>> > > >> you >>> > > >> >> want >>> > > >> >> >> > to add your own login modules, SASL server or SASL client >>> > > >> >> implementation >>> > > >> >> >> > for a mechanism that is not supported in your JDK, you >>> can do >>> > > that >>> > > >> by >>> > > >> >> >> > adding new Java security modules, using the built-in >>> > > pluggability >>> > > >> of >>> > > >> >> Java >>> > > >> >> >> > security. >>> > > >> >> >> > >>> > > >> >> >> > And finally, if you want to enable multiple mechanisms in >>> your >>> > > >> Kafka >>> > > >> >> >> > brokers, you can specify a callback handler class that >>> either >>> > > >> provides >>> > > >> >> >> the >>> > > >> >> >> > handlers for all the mechanisms (as the PR does for >>> Kerberos and >>> > > >> >> PLAIN) >>> > > >> >> >> or >>> > > >> >> >> > a delegator that calls different callback handlers based >>> on the >>> > > >> >> >> mechanism. >>> > > >> >> >> > And for login, you provide a single or a compound class >>> that >>> > > >> provides >>> > > >> >> any >>> > > >> >> >> > additional login logic for all the login modules that you >>> have >>> > > >> >> specified >>> > > >> >> >> > for KafkaServer. >>> > > >> >> >> > >>> > > >> >> >> > Hope that helps. >>> > > >> >> >> > >>> > > >> >> >> > >>> > > >> >> >> > On Tue, Mar 8, 2016 at 1:17 AM, Gwen Shapira < >>> g...@confluent.io >>> > > > >>> > > >> >> wrote: >>> > > >> >> >> > >>> > > >> >> >> >> Can you explain the process for a adding a new mechanism >>> based >>> > > on >>> > > >> >> >> current >>> > > >> >> >> >> KIP? >>> > > >> >> >> >> >>> > > >> >> >> >> My thought is that if it requires modifying Apache Kafka >>> code, >>> > > it >>> > > >> is >>> > > >> >> >> >> not pluggable enough. >>> > > >> >> >> >> >>> > > >> >> >> >> On Mon, Mar 7, 2016 at 4:04 PM, Rajini Sivaram >>> > > >> >> >> >> <rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> >> > Gwen, >>> > > >> >> >> >> > >>> > > >> >> >> >> > In cases where you want completely different callbacks >>> for >>> > > >> >> different >>> > > >> >> >> >> > mechanisms, I was thinking that the choice would be >>> between a >>> > > >> map >>> > > >> >> of >>> > > >> >> >> >> > classes (mechanism -> callbackHandler class) or a >>> delegator >>> > > >> class >>> > > >> >> that >>> > > >> >> >> >> > chooses the appropriate callback handler class based on >>> > > >> mechanism. >>> > > >> >> I >>> > > >> >> >> >> chose >>> > > >> >> >> >> > the latter since it makes it easier to configure in >>> Kafka. >>> > > >> Since we >>> > > >> >> >> >> create >>> > > >> >> >> >> > a callback handler for each channel and configure it >>> with the >>> > > >> >> >> >> > client-selected mechanism, it is straightforward to >>> have one >>> > > >> >> wrapper >>> > > >> >> >> >> class >>> > > >> >> >> >> > that delegates to the right mechanism-specific class to >>> > > handle >>> > > >> >> >> callbacks. >>> > > >> >> >> >> > In many cases, a single class may be sufficient (the >>> PR uses >>> > > a >>> > > >> >> single >>> > > >> >> >> >> > callback class for Kerberos and PLAIN). I do see your >>> point >>> > > >> about >>> > > >> >> the >>> > > >> >> >> >> > flexibility that multiple classes would provide, but >>> since >>> > > you >>> > > >> need >>> > > >> >> >> to be >>> > > >> >> >> >> > able to associate the callback with a mechanism for >>> this to >>> > > be >>> > > >> >> >> useful, I >>> > > >> >> >> >> am >>> > > >> >> >> >> > not sure if just a list would add value. >>> > > >> >> >> >> > >>> > > >> >> >> >> > Login class is slightly different since the proposal >>> is to >>> > > use a >>> > > >> >> >> single >>> > > >> >> >> >> > login context with multiple login modules to handle >>> multiple >>> > > >> >> >> mechanisms. >>> > > >> >> >> >> In >>> > > >> >> >> >> > this case, you want to perform login for all the >>> mechanisms >>> > > that >>> > > >> >> are >>> > > >> >> >> >> > enabled. And you want to call loginContext.login() >>> only once. >>> > > >> >> Again, >>> > > >> >> >> you >>> > > >> >> >> >> > can delegate to multiple classes if you wish to add >>> some >>> > > complex >>> > > >> >> >> >> > mechanism-specific logic, but a single login class >>> makes the >>> > > >> >> mapping >>> > > >> >> >> to a >>> > > >> >> >> >> > single login context and the login cache more obvious >>> (the PR >>> > > >> has a >>> > > >> >> >> test >>> > > >> >> >> >> > that includes Kerberos and PLAIN). >>> > > >> >> >> >> > >>> > > >> >> >> >> > Thoughts? >>> > > >> >> >> >> > >>> > > >> >> >> >> > On Mon, Mar 7, 2016 at 9:57 PM, Gwen Shapira < >>> > > g...@confluent.io >>> > > >> > >>> > > >> >> >> wrote: >>> > > >> >> >> >> > >>> > > >> >> >> >> >> Hi, >>> > > >> >> >> >> >> >>> > > >> >> >> >> >> Before I can vote on this KIP, I have two additional >>> > > questions >>> > > >> / >>> > > >> >> >> >> >> comments on the new configuration: >>> > > >> >> >> >> >> >>> > > >> >> >> >> >> 1. sasl.callback.handler.class - it looks like we >>> want a >>> > > single >>> > > >> >> class >>> > > >> >> >> >> >> that implements all mechanisms. I think this will >>> make it >>> > > >> >> difficult >>> > > >> >> >> to >>> > > >> >> >> >> >> extend since the only way I can add a mechanism will >>> be by >>> > > >> >> >> >> >> implementing every single existing mechanism >>> (otherwise >>> > > >> customers >>> > > >> >> >> will >>> > > >> >> >> >> >> need to choose between new and existing when >>> selecting which >>> > > >> >> class to >>> > > >> >> >> >> >> use). If Microsoft releases a proprietary "AD >>> Mechanism" and >>> > > >> >> Oracle >>> > > >> >> >> >> >> releases "OID mechanism", there will be no class that >>> > > >> implements >>> > > >> >> >> both. >>> > > >> >> >> >> >> Can we make it a list of classes instead? I realize >>> this >>> > > >> >> complicates >>> > > >> >> >> >> >> the code a bit (some kind of factory will be required >>> to >>> > > choose >>> > > >> >> the >>> > > >> >> >> >> >> right class to use), but important IMO. >>> > > >> >> >> >> >> 2. similar for sasl.login.class - if I have a class >>> for >>> > > >> Kerberos >>> > > >> >> >> (with >>> > > >> >> >> >> >> refresh thread) and a class for "plain", we need to >>> be able >>> > > to >>> > > >> >> load >>> > > >> >> >> >> >> both. >>> > > >> >> >> >> >> >>> > > >> >> >> >> >> Gwen >>> > > >> >> >> >> >> >>> > > >> >> >> >> >> On Wed, Mar 2, 2016 at 12:30 AM, Rajini Sivaram >>> > > >> >> >> >> >> <rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> >> >> > Jun, >>> > > >> >> >> >> >> > >>> > > >> >> >> >> >> > Thanks, I have added a note to the KIP. I will add a >>> > > comment >>> > > >> in >>> > > >> >> the >>> > > >> >> >> >> >> > implementation and also add a unit test to ensure >>> that >>> > > >> conflicts >>> > > >> >> >> are >>> > > >> >> >> >> >> > avoided when version number is modified. >>> > > >> >> >> >> >> > >>> > > >> >> >> >> >> > On Tue, Mar 1, 2016 at 5:43 PM, Jun Rao < >>> j...@confluent.io >>> > > > >>> > > >> >> wrote: >>> > > >> >> >> >> >> > >>> > > >> >> >> >> >> >> Rajini, >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >> Thanks for the explanation. For 1, this implies >>> that we >>> > > have >>> > > >> >> to be >>> > > >> >> >> >> >> careful >>> > > >> >> >> >> >> >> with changing the 2-byte version in the future to >>> avoid >>> > > >> >> conflict. >>> > > >> >> >> >> Could >>> > > >> >> >> >> >> you >>> > > >> >> >> >> >> >> document this in the KIP and also in the >>> implementation? >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >> Jun >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >> On Tue, Mar 1, 2016 at 2:47 AM, Rajini Sivaram < >>> > > >> >> >> >> >> >> rajinisiva...@googlemail.com >>> > > >> >> >> >> >> >> > wrote: >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >> > Jun, >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > Thank you for the review. >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > 1. With GSSAPI, the first context >>> establishment >>> > > packet >>> > > >> >> starts >>> > > >> >> >> >> with >>> > > >> >> >> >> >> the >>> > > >> >> >> >> >> >> > byte 0x60 (tag for APPLICATION-0) followed by >>> a >>> > > >> >> >> variable-length >>> > > >> >> >> >> >> >> encoded >>> > > >> >> >> >> >> >> > size, followed by various tags and contents. >>> And the >>> > > >> >> packet >>> > > >> >> >> also >>> > > >> >> >> >> >> >> > contains a >>> > > >> >> >> >> >> >> > checksum. This is completely different from >>> the >>> > > >> mechanism >>> > > >> >> >> packet >>> > > >> >> >> >> >> from >>> > > >> >> >> >> >> >> > Kafka >>> > > >> >> >> >> >> >> > clients which start with a two-byte version >>> set to >>> > > zero >>> > > >> >> >> >> currently, >>> > > >> >> >> >> >> >> > followed >>> > > >> >> >> >> >> >> > by just a String mechanism. >>> > > >> >> >> >> >> >> > 2. Agreed, I have removed the version from the >>> > > server >>> > > >> >> >> response >>> > > >> >> >> >> in >>> > > >> >> >> >> >> the >>> > > >> >> >> >> >> >> > KIP. Thanks. >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > On Tue, Mar 1, 2016 at 2:33 AM, Jun Rao < >>> > > j...@confluent.io >>> > > >> > >>> > > >> >> >> wrote: >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> > > Rajini, >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > Thanks for the updates. Just a couple of minor >>> > > comments. >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > 1. With the default GSSAPI, what's the first >>> packet >>> > > that >>> > > >> >> the >>> > > >> >> >> >> client >>> > > >> >> >> >> >> >> sends >>> > > >> >> >> >> >> >> > > to the server? Is that completely different >>> from the >>> > > >> packet >>> > > >> >> >> >> format >>> > > >> >> >> >> >> that >>> > > >> >> >> >> >> >> > we >>> > > >> >> >> >> >> >> > > will use for non-GSSAPI mechanisms? >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > 2. In the server response, it doesn't seem >>> that we >>> > > need >>> > > >> to >>> > > >> >> >> >> include >>> > > >> >> >> >> >> the >>> > > >> >> >> >> >> >> > > version since the client knows the version of >>> the >>> > > >> request >>> > > >> >> >> that it >>> > > >> >> >> >> >> >> sends. >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > Jun >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > On Mon, Feb 29, 2016 at 10:14 AM, Rajini >>> Sivaram < >>> > > >> >> >> >> >> >> > > rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > > > Harsha, >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > > Thank you for the review. I will wait >>> another day >>> > > to >>> > > >> see >>> > > >> >> if >>> > > >> >> >> >> there >>> > > >> >> >> >> >> is >>> > > >> >> >> >> >> >> > more >>> > > >> >> >> >> >> >> > > > feedback and then start a voting thread. >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > > Rajini >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > > On Mon, Feb 29, 2016 at 2:51 PM, Harsha < >>> > > >> ka...@harsha.io >>> > > >> >> > >>> > > >> >> >> >> wrote: >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > > > Rajini, >>> > > >> >> >> >> >> >> > > > > Thanks for the changes to >>> the KIP. >>> > > It >>> > > >> >> looks >>> > > >> >> >> >> good >>> > > >> >> >> >> >> to >>> > > >> >> >> >> >> >> > me. I >>> > > >> >> >> >> >> >> > > > > think we can move to voting. >>> > > >> >> >> >> >> >> > > > > Thanks, >>> > > >> >> >> >> >> >> > > > > Harsha >>> > > >> >> >> >> >> >> > > > > >>> > > >> >> >> >> >> >> > > > > On Mon, Feb 29, 2016, at 12:43 AM, Rajini >>> Sivaram >>> > > >> >> wrote: >>> > > >> >> >> >> >> >> > > > > > I have added some more detail to the KIP >>> based >>> > > on >>> > > >> the >>> > > >> >> >> >> >> discussion >>> > > >> >> >> >> >> >> in >>> > > >> >> >> >> >> >> > > the >>> > > >> >> >> >> >> >> > > > > > last KIP meeting to simplify support for >>> > > multiple >>> > > >> >> >> >> mechanisms. >>> > > >> >> >> >> >> >> Have >>> > > >> >> >> >> >> >> > > also >>> > > >> >> >> >> >> >> > > > > > changed the property names to reflect >>> this. >>> > > >> >> >> >> >> >> > > > > > >>> > > >> >> >> >> >> >> > > > > > Also updated the PR in >>> > > >> >> >> >> >> >> > > > >>> https://issues.apache.org/jira/browse/KAFKA-3149 >>> > > >> >> >> >> >> >> > > > > > to >>> > > >> >> >> >> >> >> > > > > > reflect the KIP. >>> > > >> >> >> >> >> >> > > > > > >>> > > >> >> >> >> >> >> > > > > > Any feedback is appreciated. >>> > > >> >> >> >> >> >> > > > > > >>> > > >> >> >> >> >> >> > > > > > >>> > > >> >> >> >> >> >> > > > > > On Tue, Feb 23, 2016 at 9:36 PM, Rajini >>> > > Sivaram < >>> > > >> >> >> >> >> >> > > > > > rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > > >> >> >> >> >> >> > > > > > > I have updated the KIP based on the >>> > > discussion >>> > > >> in >>> > > >> >> the >>> > > >> >> >> KIP >>> > > >> >> >> >> >> >> meeting >>> > > >> >> >> >> >> >> > > > > today. >>> > > >> >> >> >> >> >> > > > > > > >>> > > >> >> >> >> >> >> > > > > > > Comments and feedback are welcome. >>> > > >> >> >> >> >> >> > > > > > > >>> > > >> >> >> >> >> >> > > > > > > On Wed, Feb 3, 2016 at 7:20 PM, Rajini >>> > > Sivaram < >>> > > >> >> >> >> >> >> > > > > > > rajinisiva...@googlemail.com> wrote: >>> > > >> >> >> >> >> >> > > > > > > >>> > > >> >> >> >> >> >> > > > > > >> Hi Harsha, >>> > > >> >> >> >> >> >> > > > > > >> >>> > > >> >> >> >> >> >> > > > > > >> Thank you for the review. Can you >>> clarify - >>> > > I >>> > > >> >> think >>> > > >> >> >> you >>> > > >> >> >> >> are >>> > > >> >> >> >> >> >> > saying >>> > > >> >> >> >> >> >> > > > > that >>> > > >> >> >> >> >> >> > > > > > >> the client should send its mechanism >>> over >>> > > the >>> > > >> >> wire to >>> > > >> >> >> >> the >>> > > >> >> >> >> >> >> > server. >>> > > >> >> >> >> >> >> > > Is >>> > > >> >> >> >> >> >> > > > > that >>> > > >> >> >> >> >> >> > > > > > >> correct? The exchange is slightly >>> different >>> > > in >>> > > >> the >>> > > >> >> >> KIP >>> > > >> >> >> >> >> (the PR >>> > > >> >> >> >> >> >> > > > > matches the >>> > > >> >> >> >> >> >> > > > > > >> KIP) from the one you described to >>> enable >>> > > >> >> >> >> interoperability >>> > > >> >> >> >> >> >> with >>> > > >> >> >> >> >> >> > > > > 0.9.0.0. >>> > > >> >> >> >> >> >> > > > > > >> >>> > > >> >> >> >> >> >> > > > > > >> >>> > > >> >> >> >> >> >> > > > > > >> On Wed, Feb 3, 2016 at 1:56 PM, >>> Harsha < >>> > > >> >> >> m...@harsha.io> >>> > > >> >> >> >> >> >> wrote: >>> > > >> >> >> >> >> >> > > > > > >> >>> > > >> >> >> >> >> >> > > > > > >>> Rajini, >>> > > >> >> >> >> >> >> > > > > > >>> I looked at the PR you >>> have. I >>> > > >> think >>> > > >> >> its >>> > > >> >> >> >> better >>> > > >> >> >> >> >> >> with >>> > > >> >> >> >> >> >> > > > your >>> > > >> >> >> >> >> >> > > > > > >>> earlier approach rather >>> than >>> > > >> extending >>> > > >> >> >> the >>> > > >> >> >> >> >> >> protocol. >>> > > >> >> >> >> >> >> > > > > > >>> What I was thinking initially is, >>> Broker >>> > > has a >>> > > >> >> >> config >>> > > >> >> >> >> >> option >>> > > >> >> >> >> >> >> of >>> > > >> >> >> >> >> >> > > say >>> > > >> >> >> >> >> >> > > > > > >>> sasl.mechanism = GSSAPI, PLAIN >>> > > >> >> >> >> >> >> > > > > > >>> and the client can have similar >>> config of >>> > > >> >> >> >> >> >> sasl.mechanism=PLAIN. >>> > > >> >> >> >> >> >> > > > > Client >>> > > >> >> >> >> >> >> > > > > > >>> can send its sasl mechanism before >>> the >>> > > >> handshake >>> > > >> >> >> starts >>> > > >> >> >> >> >> and >>> > > >> >> >> >> >> >> if >>> > > >> >> >> >> >> >> > > the >>> > > >> >> >> >> >> >> > > > > > >>> broker accepts that particular >>> mechanism >>> > > than >>> > > >> it >>> > > >> >> >> can go >>> > > >> >> >> >> >> ahead >>> > > >> >> >> >> >> >> > > with >>> > > >> >> >> >> >> >> > > > > > >>> handshake otherwise return a error >>> saying >>> > > that >>> > > >> >> the >>> > > >> >> >> >> >> mechanism >>> > > >> >> >> >> >> >> > not >>> > > >> >> >> >> >> >> > > > > > >>> allowed. >>> > > >> >> >> >> >> >> > > > > > >>> >>> > > >> >> >> >> >> >> > > > > > >>> Thanks, >>> > > >> >> >> >> >> >> > > > > > >>> Harsha >>> > > >> >> >> >> >> >> > > > > > >>> >>> > > >> >> >> >> >> >> > > > > > >>> On Wed, Feb 3, 2016, at 04:58 AM, >>> Rajini >>> > > >> Sivaram >>> > > >> >> >> wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > A slightly different approach for >>> > > supporting >>> > > >> >> >> >> different >>> > > >> >> >> >> >> SASL >>> > > >> >> >> >> >> >> > > > > mechanisms >>> > > >> >> >> >> >> >> > > > > > >>> > within a broker is to allow the >>> same >>> > > >> "*security >>> > > >> >> >> >> >> protocol*" >>> > > >> >> >> >> >> >> to >>> > > >> >> >> >> >> >> > > be >>> > > >> >> >> >> >> >> > > > > used >>> > > >> >> >> >> >> >> > > > > > >>> on >>> > > >> >> >> >> >> >> > > > > > >>> > different ports with different >>> > > configuration >>> > > >> >> >> >> options. An >>> > > >> >> >> >> >> >> > > > advantage >>> > > >> >> >> >> >> >> > > > > of >>> > > >> >> >> >> >> >> > > > > > >>> > this >>> > > >> >> >> >> >> >> > > > > > >>> > approach is that it extends the >>> > > >> >> configurability of >>> > > >> >> >> >> not >>> > > >> >> >> >> >> just >>> > > >> >> >> >> >> >> > > SASL, >>> > > >> >> >> >> >> >> > > > > but >>> > > >> >> >> >> >> >> > > > > > >>> any >>> > > >> >> >> >> >> >> > > > > > >>> > protocol. For instance, it would >>> enable >>> > > the >>> > > >> >> use of >>> > > >> >> >> >> SSL >>> > > >> >> >> >> >> with >>> > > >> >> >> >> >> >> > > > mutual >>> > > >> >> >> >> >> >> > > > > > >>> client >>> > > >> >> >> >> >> >> > > > > > >>> > authentication on one port or >>> different >>> > > >> >> >> certificate >>> > > >> >> >> >> >> chains >>> > > >> >> >> >> >> >> on >>> > > >> >> >> >> >> >> > > > > another. >>> > > >> >> >> >> >> >> > > > > > >>> > And >>> > > >> >> >> >> >> >> > > > > > >>> > it avoids the need for SASL >>> mechanism >>> > > >> >> negotiation. >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > Kafka would have the same >>> "*security >>> > > >> protocols" >>> > > >> >> >> >> >> *defined as >>> > > >> >> >> >> >> >> > > > today, >>> > > >> >> >> >> >> >> > > > > but >>> > > >> >> >> >> >> >> > > > > > >>> > with >>> > > >> >> >> >> >> >> > > > > > >>> > (a single) configurable SASL >>> mechanism. >>> > > To >>> > > >> have >>> > > >> >> >> >> >> different >>> > > >> >> >> >> >> >> > > > > > >>> configurations >>> > > >> >> >> >> >> >> > > > > > >>> > of >>> > > >> >> >> >> >> >> > > > > > >>> > a protocol within a broker, users >>> can >>> > > define >>> > > >> >> new >>> > > >> >> >> >> >> protocol >>> > > >> >> >> >> >> >> > names >>> > > >> >> >> >> >> >> > > > > which >>> > > >> >> >> >> >> >> > > > > > >>> are >>> > > >> >> >> >> >> >> > > > > > >>> > configured versions of existing >>> > > protocols, >>> > > >> >> perhaps >>> > > >> >> >> >> using >>> > > >> >> >> >> >> >> just >>> > > >> >> >> >> >> >> > > > > > >>> > configuration >>> > > >> >> >> >> >> >> > > > > > >>> > entries and no additional code. >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > For example: >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > A single mechanism broker would be >>> > > >> configured >>> > > >> >> as: >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > listeners=SASL_SSL://:9092 >>> > > >> >> >> >> >> >> > > > > > >>> > sasl.mechanism=GSSAPI >>> > > >> >> >> >> >> >> > > > > > >>> > sasl.kerberos.class.name=kafka >>> > > >> >> >> >> >> >> > > > > > >>> > ... >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > And a multi-mechanism broker would >>> be >>> > > >> >> configured >>> > > >> >> >> as: >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> listeners=gssapi://:9092,plain://:9093,custom://:9094 >>> > > >> >> >> >> >> >> > > > > > >>> > gssapi.security.protocol=SASL_SSL >>> > > >> >> >> >> >> >> > > > > > >>> > gssapi.sasl.mechanism=GSSAPI >>> > > >> >> >> >> >> >> > > > > > >>> > gssapi.sasl.kerberos.class.name >>> =kafka >>> > > >> >> >> >> >> >> > > > > > >>> > ... >>> > > >> >> >> >> >> >> > > > > > >>> > plain.security.protocol=SASL_SSL >>> > > >> >> >> >> >> >> > > > > > >>> > plain.sasl.mechanism=PLAIN >>> > > >> >> >> >> >> >> > > > > > >>> > .. >>> > > >> >> >> >> >> >> > > > > > >>> > >>> custom.security.protocol=SASL_PLAINTEXT >>> > > >> >> >> >> >> >> > > > > > >>> > custom.sasl.mechanism=CUSTOM >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >>> custom.sasl.callback.handler.class=example.CustomCallbackHandler >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > This is still a big change because >>> it >>> > > >> affects >>> > > >> >> the >>> > > >> >> >> >> >> currently >>> > > >> >> >> >> >> >> > > fixed >>> > > >> >> >> >> >> >> > > > > > >>> > enumeration of security protocol >>> > > >> definitions, >>> > > >> >> but >>> > > >> >> >> one >>> > > >> >> >> >> >> that >>> > > >> >> >> >> >> >> is >>> > > >> >> >> >> >> >> > > > > perhaps >>> > > >> >> >> >> >> >> > > > > > >>> > more >>> > > >> >> >> >> >> >> > > > > > >>> > flexible than defining every new >>> SASL >>> > > >> mechanism >>> > > >> >> >> as a >>> > > >> >> >> >> new >>> > > >> >> >> >> >> >> > > security >>> > > >> >> >> >> >> >> > > > > > >>> > protocol. >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > Thoughts? >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > On Tue, Feb 2, 2016 at 12:20 PM, >>> Rajini >>> > > >> >> Sivaram < >>> > > >> >> >> >> >> >> > > > > > >>> > rajinisiva...@googlemail.com> >>> wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > >>> > > >> >> >> >> >> >> > > > > > >>> > > As Ismael has said, we do not >>> have a >>> > > >> >> >> requirement to >>> > > >> >> >> >> >> >> support >>> > > >> >> >> >> >> >> > > > > multiple >>> > > >> >> >> >> >> >> > > > > > >>> > > protocols in a broker. But I >>> agree with >>> > > >> Jun's >>> > > >> >> >> >> >> observation >>> > > >> >> >> >> >> >> > > that >>> > > >> >> >> >> >> >> > > > > some >>> > > >> >> >> >> >> >> > > > > > >>> > > companies might want to support a >>> > > >> different >>> > > >> >> >> >> >> >> authentication >>> > > >> >> >> >> >> >> > > > > mechanism >>> > > >> >> >> >> >> >> > > > > > >>> for >>> > > >> >> >> >> >> >> > > > > > >>> > > internal users or partners. For >>> > > instance, >>> > > >> we >>> > > >> >> do >>> > > >> >> >> use >>> > > >> >> >> >> >> two >>> > > >> >> >> >> >> >> > > > different >>> > > >> >> >> >> >> >> > > > > > >>> > > authentication mechanisms, it >>> just so >>> > > >> happens >>> > > >> >> >> that >>> > > >> >> >> >> we >>> > > >> >> >> >> >> are >>> > > >> >> >> >> >> >> > > able >>> > > >> >> >> >> >> >> > > > > to use >>> > > >> >> >> >> >> >> > > > > > >>> > > certificate-based authentication >>> for >>> > > >> internal >>> > > >> >> >> >> users, >>> > > >> >> >> >> >> and >>> > > >> >> >> >> >> >> > > hence >>> > > >> >> >> >> >> >> > > > > don't >>> > > >> >> >> >> >> >> > > > > > >>> > > require multiple SASL mechanisms >>> in a >>> > > >> broker. >>> > > >> >> >> >> >> >> > > > > > >>> > > >>> > > >> >> >> >> >> >> > > > > > >>> > > As Tao has pointed out, mechanism >>> > > >> negotiation >>> > > >> >> >> is a >>> > > >> >> >> >> >> common >>> > > >> >> >> >> >> >> > > usage >>> > > >> >> >> >> >> >> > > > > > >>> pattern. >>> > > >> >> >> >> >> >> > > > > > >>> > > Many existing protocols that >>> support >>> > > SASL >>> > > >> do >>> > > >> >> >> >> already >>> > > >> >> >> >> >> use >>> > > >> >> >> >> >> >> > this >>> > > >> >> >> >> >> >> > > > > > >>> pattern. AMQP >>> > > >> >> >> >> >> >> > > > > > >>> > > ( >>> > > >> >> >> >> >> >> > > > > > >>> > > >>> > > >> >> >> >> >> >> > > > > > >>> >>> > > >> >> >> >> >> >> > > > > >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >>> > > >> >> >> >> >>> > > >> >> >> >>> > > >> >> >>> > > >> >>> > > >>> http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-security-v1.0-os.html#type-sasl-mechanisms >>> > > >> >> >> >> >> >> > > > > > >>> ), >>> > > >> >> >> >> >> >> > > > > > >>> > > which, as a messaging protocol >>> maybe >>> > > >> closer >>> > > >> >> to >>> > > >> >> >> >> Kafka >>> > > >> >> >> >> >> in >>> > > >> >> >> >> >> >> use >>> > > >> >> >> >> >> >> > > > cases >>> > > >> >> >> >> >> >> > > > > > >>> than >>> > > >> >> >> >> >> >> > > > > > >>> > > Zookeeper, is an example. Other >>> > > examples >>> > > >> >> where >>> > > >> >> >> the >>> > > >> >> >> >> >> client >>> > > >> >> >> >> >> >> > > > > negotiates >>> > > >> >> >> >> >> >> > > > > > >>> or >>> > > >> >> >> >> >> >> > > > > > >>> > > sends SASL mechanism to server >>> include >>> > > >> ACAP >>> > > >> >> >> that is >>> > > >> >> >> >> >> used >>> > > >> >> >> >> >> >> as >>> > > >> >> >> >> >> >> > > an >>> > > >> >> >> >> >> >> > > > > > >>> example in >>> > > >> >> >> >> >> >> > > > > > >>> > > the SASL RFCs, POP3, LDAP, SMTP >>> etc. >>> > > This >>> > > >> is >>> > > >> >> >> not to >>> > > >> >> >> >> >> say >>> > > >> >> >> >> >> >> > that >>> > > >> >> >> >> >> >> > > > > Kafka >>> > > >> >> >> >> >> >> > > > > > >>> > > shouldn't use a different type of >>> > > >> mechanism >>> > > >> >> >> >> selection >>> > > >> >> >> >> >> >> that >>> > > >> >> >> >> >> >> > > fits >>> > > >> >> >> >> >> >> > > > > > >>> better with >>> > > >> >> >> >> >> >> > > > > > >>> > > the existing Kafka design. Just >>> that >>> > > >> >> negotiation >>> > > >> >> >> >> is a >>> > > >> >> >> >> >> >> > common >>> > > >> >> >> >> >> >> > > > > pattern >>> > > >> >> >> >> >> >> > > > > > >>> and >>> > > >> >> >> >> >> >> > > > > > >>> > > since we typically turn on >>> > > >> javax.net.debug to >>> > > >> >> >> debug >>> > > >> >> >> >> >> TLS >>> > > >> >> >> >> >> >> > > > > negotiation >>> > > >> >> >> >> >> >> > > > > > >>> issues, >>> > > >> >> >> >> >> >> > > > > > >>> > > having to use Kafka logging to >>> debug >>> > > SASL >>> > > >> >> >> >> negotiation >>> > > >> >> >> >> >> >> > issues >>> > > >> >> >> >> >> >> > > is >>> > > >> >> >> >> >> >> > > > > not >>> > > >> >> >> >> >> >> > > > > > >>> that >>> > > >> >> >> >> >> >> > > > > > >>> > > dissimilar. >>> > > >> >> >> >> >> >> > > > > > >>> > > >>> > > >> >> >> >> >> >> > > > > > >>> > > >>> > > >> >> >> >> >> >> > > > > > >>> > > On Tue, Feb 2, 2016 at 6:12 AM, >>> tao >>> > > xiao < >>> > > >> >> >> >> >> >> > > xiaotao...@gmail.com >>> > > >> >> >> >> >> >> > > > > >>> > > >> >> >> >> >> >> > > > > > >>> wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> I am the author of KIP-44. I >>> hope my >>> > > use >>> > > >> >> case >>> > > >> >> >> will >>> > > >> >> >> >> >> add >>> > > >> >> >> >> >> >> > some >>> > > >> >> >> >> >> >> > > > > values >>> > > >> >> >> >> >> >> > > > > > >>> to this >>> > > >> >> >> >> >> >> > > > > > >>> > >> discussion. The reason I raised >>> KIP44 >>> > > is >>> > > >> >> that I >>> > > >> >> >> >> want >>> > > >> >> >> >> >> to >>> > > >> >> >> >> >> >> be >>> > > >> >> >> >> >> >> > > > able >>> > > >> >> >> >> >> >> > > > > to >>> > > >> >> >> >> >> >> > > > > > >>> > >> implement a custom security >>> protocol >>> > > that >>> > > >> >> can >>> > > >> >> >> >> fulfill >>> > > >> >> >> >> >> >> the >>> > > >> >> >> >> >> >> > > need >>> > > >> >> >> >> >> >> > > > > of my >>> > > >> >> >> >> >> >> > > > > > >>> > >> company. As pointed out by >>> Ismael >>> > > KIP-43 >>> > > >> now >>> > > >> >> >> >> >> supports a >>> > > >> >> >> >> >> >> > > > > pluggable >>> > > >> >> >> >> >> >> > > > > > >>> way to >>> > > >> >> >> >> >> >> > > > > > >>> > >> inject custom security provider >>> to >>> > > SASL I >>> > > >> >> think >>> > > >> >> >> >> it is >>> > > >> >> >> >> >> >> > enough >>> > > >> >> >> >> >> >> > > > to >>> > > >> >> >> >> >> >> > > > > > >>> cover the >>> > > >> >> >> >> >> >> > > > > > >>> > >> use case I have and address the >>> > > concerns >>> > > >> >> >> raised in >>> > > >> >> >> >> >> >> KIP-44. >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> > >> For multiple security protocols >>> > > support >>> > > >> >> >> >> >> simultaneously >>> > > >> >> >> >> >> >> it >>> > > >> >> >> >> >> >> > is >>> > > >> >> >> >> >> >> > > > not >>> > > >> >> >> >> >> >> > > > > > >>> needed in >>> > > >> >> >> >> >> >> > > > > > >>> > >> my use case and I don't foresee >>> it is >>> > > >> >> needed in >>> > > >> >> >> >> the >>> > > >> >> >> >> >> >> future >>> > > >> >> >> >> >> >> > > but >>> > > >> >> >> >> >> >> > > > > as i >>> > > >> >> >> >> >> >> > > > > > >>> said >>> > > >> >> >> >> >> >> > > > > > >>> > >> this is my use case only there >>> may be >>> > > >> other >>> > > >> >> use >>> > > >> >> >> >> cases >>> > > >> >> >> >> >> >> that >>> > > >> >> >> >> >> >> > > > need >>> > > >> >> >> >> >> >> > > > > it. >>> > > >> >> >> >> >> >> > > > > > >>> But if >>> > > >> >> >> >> >> >> > > > > > >>> > >> we want to support it in the >>> future I >>> > > >> >> prefer to >>> > > >> >> >> >> get >>> > > >> >> >> >> >> it >>> > > >> >> >> >> >> >> > right >>> > > >> >> >> >> >> >> > > > at >>> > > >> >> >> >> >> >> > > > > the >>> > > >> >> >> >> >> >> > > > > > >>> first >>> > > >> >> >> >> >> >> > > > > > >>> > >> place given the fact that >>> security >>> > > >> protocol >>> > > >> >> is >>> > > >> >> >> an >>> > > >> >> >> >> >> ENUM >>> > > >> >> >> >> >> >> and >>> > > >> >> >> >> >> >> > > if >>> > > >> >> >> >> >> >> > > > we >>> > > >> >> >> >> >> >> > > > > > >>> stick to >>> > > >> >> >> >> >> >> > > > > > >>> > >> that implementation it is very >>> hard to >>> > > >> >> extend >>> > > >> >> >> in >>> > > >> >> >> >> the >>> > > >> >> >> >> >> >> > future >>> > > >> >> >> >> >> >> > > > > when we >>> > > >> >> >> >> >> >> > > > > > >>> decide >>> > > >> >> >> >> >> >> > > > > > >>> > >> multiple security protocols is >>> needed. >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> > >> Protocol negotiation is a very >>> common >>> > > >> usage >>> > > >> >> >> >> pattern >>> > > >> >> >> >> >> in >>> > > >> >> >> >> >> >> > > > security >>> > > >> >> >> >> >> >> > > > > > >>> domain. As >>> > > >> >> >> >> >> >> > > > > > >>> > >> suggested in Java SASL doc >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> >>> > > >> >> >> >> >> >> > > > > >>> > > >> >> >> >> >> >> > > > >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> > >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >>> > > >> >> >> >> >>> > > >> >> >> >>> > > >> >> >>> > > >> >>> > > >>> http://docs.oracle.com/javase/7/docs/technotes/guides/security/sasl/sasl-refguide.html >>> > > >> >> >> >> >> >> > > > > > >>> > >> client >>> > > >> >> >> >> >> >> > > > > > >>> > >> first sends out a packet to >>> server and >>> > > >> >> server >>> > > >> >> >> >> >> responds >>> > > >> >> >> >> >> >> > with >>> > > >> >> >> >> >> >> > > a >>> > > >> >> >> >> >> >> > > > > list >>> > > >> >> >> >> >> >> > > > > > >>> of >>> > > >> >> >> >> >> >> > > > > > >>> > >> mechanisms it supports. This is >>> very >>> > > >> >> similar to >>> > > >> >> >> >> >> SSL/TLS >>> > > >> >> >> >> >> >> > > > > negotiation. >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> > >> On Tue, 2 Feb 2016 at 06:39 >>> Ismael >>> > > Juma < >>> > > >> >> >> >> >> >> > ism...@juma.me.uk> >>> > > >> >> >> >> >> >> > > > > wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> > >> > On Mon, Feb 1, 2016 at 7:04 >>> PM, Gwen >>> > > >> >> Shapira >>> > > >> >> >> < >>> > > >> >> >> >> >> >> > > > > g...@confluent.io> >>> > > >> >> >> >> >> >> > > > > > >>> wrote: >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > Looking at "existing >>> solutions", >>> > > it >>> > > >> >> looks >>> > > >> >> >> like >>> > > >> >> >> >> >> >> > Zookeeper >>> > > >> >> >> >> >> >> > > > > allows >>> > > >> >> >> >> >> >> > > > > > >>> > >> plugging >>> > > >> >> >> >> >> >> > > > > > >>> > >> > in >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > any SASL mechanism, but the >>> server >>> > > >> will >>> > > >> >> >> only >>> > > >> >> >> >> >> support >>> > > >> >> >> >> >> >> > one >>> > > >> >> >> >> >> >> > > > > > >>> mechanism at >>> > > >> >> >> >> >> >> > > > > > >>> > >> a >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > time. >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > This was the original >>> proposal from >>> > > >> >> Rajini as >>> > > >> >> >> >> that >>> > > >> >> >> >> >> is >>> > > >> >> >> >> >> >> > > enough >>> > > >> >> >> >> >> >> > > > > for >>> > > >> >> >> >> >> >> > > > > > >>> their >>> > > >> >> >> >> >> >> > > > > > >>> > >> > needs. >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > If this is good enough for >>> our >>> > > >> use-case >>> > > >> >> >> (do we >>> > > >> >> >> >> >> >> > actually >>> > > >> >> >> >> >> >> > > > > need to >>> > > >> >> >> >> >> >> > > > > > >>> > >> support >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > multiple mechanisms at >>> once?), it >>> > > >> will >>> > > >> >> >> >> simplify >>> > > >> >> >> >> >> >> life a >>> > > >> >> >> >> >> >> > > lot >>> > > >> >> >> >> >> >> > > > > for >>> > > >> >> >> >> >> >> > > > > > >>> us ( >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> >>> > > >> >> >> >> >> >> > > > > > >>> >>> > > >> >> >> >> >> >> > > > > >>> > > >> >> >> >> >> >> > > >>> > > >> >> >> >> >> >> >>> > > >> >> >> >> >> >>> > > >> >> >> >> >>> > > >> >> >> >>> > > >> >> >>> > > >> >>> > > >>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zookeeper+and+SASL >>> > > >> >> >> >> >> >> > > > > > >>> > >> > ) >>> > > >> >> >> >> >> >> > > > > > >>> > >> > > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > The current thinking is that >>> it >>> > > would >>> > > >> be >>> > > >> >> >> useful >>> > > >> >> >> >> to >>> > > >> >> >> >> >> >> > support >>> > > >> >> >> >> >> >> > > > > > >>> multiple SASL >>> > > >> >> >> >> >> >> > > > > > >>> > >> > mechanisms simultaneously. In >>> the >>> > > KIP >>> > > >> >> >> meeting, >>> > > >> >> >> >> Jun >>> > > >> >> >> >> >> >> > > mentioned >>> > > >> >> >> >> >> >> > > > > that >>> > > >> >> >> >> >> >> > > > > > >>> > >> companies >>> > > >> >> >> >> >> >> > > > > > >>> > >> > sometimes support additional >>> > > >> >> authentication >>> > > >> >> >> >> >> mechanisms >>> > > >> >> >> >> >> >> > for >>> > > >> >> >> >> >> >> > > > > > >>> partners, for >>> > > >> >> >> >> >> >> > > > > > >>> > >> > example. It does make things >>> more >>> > > >> >> complex, as >>> > > >> >> >> >> you >>> > > >> >> >> >> >> say, >>> > > >> >> >> >> >> >> > so >>> > > >> >> >> >> >> >> > > we >>> > > >> >> >> >> >> >> > > > > need >>> > > >> >> >> >> >> >> > > > > > >>> to be >>> > > >> >> >> >> >> >> > > > > > >>> > >> > sure the complexity is worth >>> it. >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > Two more points: >>> > > >> >> >> >> >> >> > > > > > >>> > >> > >>> > > >> >> >> >> >> >> > > > > > >>> > >> > 1. It has been suggested that >>> custom >>> > > >> >> security >>> > > >> >> >> >> >> protocol >>> > > >> >> >> >> >> >> > > > > support is >>> > > >> >> >> >> >> >> > > > > > >>> > >> needed by >>> > > >> >> >> >> >> >> > > > > > >>> > >> > some (KIP-44). Rajini enhanced >>> > > KIP-43 >>> > > >> so >>> > > >> >> >> that a >>> > > >> >> >> >> >> SASL >>> > > >> >> >> >> >> >> > > > mechanism >>> > > >> >> >> >> > >>> >> ... >> >> [Message clipped] > > > > > -- > Regards, > > Rajini > -- Regards, Rajini
