Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-30 Thread Joel Koshy
s even if the long term private key of the server is compromised. > > This > > > is not specified in SASL. > > > > > > > > > * Performance in JVM-based implementation. The SASL APIs in Java look > > like > > > the following: > > > >

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-29 Thread Gwen Shapira
t; > public byte[] unwrap(byte inBuf[], int offset, int len...) > > > > Note that there are significant overheads on copying bytes and GC. Take > > encrypting the data for example: > > > > (1) Users pass in the payload as a byte array. > > (2) The SASL implementation

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-29 Thread Sriharsha Chintalapani
gt; Our experience in HDFS is that for large amount of data AES is fairly > efficient compared to copying bytes and the GC activity. We saw long GC > pauses due to frequent allocation of large heap memory. In the end we work > around this problem by performing SASL authenticati

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-29 Thread Gwen Shapira
> efficient compared to copying bytes and the GC activity. We saw long GC > pauses due to frequent allocation of large heap memory. In the end we work > around this problem by performing SASL authentication first and encrypting > the traffic directly by setting up an AES codecs. > > Hop

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-29 Thread Sriharsha Chintalapani
emory. In the end we work around this problem by performing SASL authentication first and encrypting the traffic directly by setting up an AES codecs. Hope it helps. Regards, Haohui From: Sriharsha Chintalapani Sent: Tuesday, April 28, 2015 4:31 PM To: dev@kaf

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-29 Thread Rajini Sivaram
n of large heap memory. In the end we work > around this problem by performing SASL authentication first and encrypting > the traffic directly by setting up an AES codecs. > > Hope it helps. > > Regards, > Haohui > > ____________________ > > F

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-28 Thread Haohui Mai
: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation Updated KIP-12 is here https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809888 . My new proposal talks about Channel having two layers one is TransportLayer and another Authentication Layer. If users want to do authentication

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-28 Thread Sriharsha Chintalapani
Updated KIP-12 is here  https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809888 .  My new proposal talks about Channel having two layers one is TransportLayer and another Authentication Layer.    If users want to do  authentication without encryption they can use PlainTextTrans

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
I yet to update the KIP with my latest proposal. So give me few days to update it.  I am looking at supporting KERBEROS for the first release and going to use JAAS Login Modules to provide authentication. And will we provide a default SASL PLAIN mechanism on the server side  Yes . I’ll update the

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Gari Singh
Great. Sounds good. I'll re-read the KIP ASAP. Is their another KIP around authentication providers or is that being tracked here as well. For example, the SASL PLAIN mechanism carries a username and password but currently I don't know where that would be authenticated? I notice that AuthUtils

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
Hi Gari,        I apologize for not clear in KIP and starting discussion thread earlier.  My initial proposal( currently in KIP ) to provide PLAINTEXT, SSL and KERBEROS as individual protocol implementation.  As you mentioned at the end “In terms of message level integrity and confidentiality (not

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Gari Singh
Sorry for jumping in late, but I have been trying to follow this chain as well as the updates to the KIP. I don't mean to seem critical and I may be misunderstanding the proposed implementation, but there seems to be some confusion around terminology (at least from my perspective) and I am not sur

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Thank you, Harsha. Yes, that makes sense. Shall take a look when the KIP is finalized. Rajini On Fri, Apr 24, 2015 at 2:34 PM, Sriharsha Chintalapani wrote: > Rajini, > I am exploring this part right now. To support PLAINTEXT and SSL > as protocols and Kerberos auth as authentication o

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Sriharsha Chintalapani
Rajini,         I am exploring this part right now. To support PLAINTEXT and SSL as protocols and Kerberos auth as authentication on top of plaintext or ssl (if users want to do encryption over an auth mechanism). This is mainly influenced by SASL or GSS-API performance issue when I enable encry

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Have there been any discussions around separating out authentication and encryption protocols for Kafka endpoints to enable different combinations? In our deployment environment, we would like to use TLS for encryption, but we don't necessarily want to use certificate-based authentication of client

[DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-24 Thread Rajini Sivaram
Harsha, Thank you for the quick response. (Sorry had missed sending this reply to the dev-list earlier).. 1. I am not sure what the new server-side code is going to look like after refactoring under KAFKA-1928. But I was assuming that there would be only one Channel implementation tha

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-22 Thread Sriharsha Chintalapani
1. *Support for running potentially long-running delegated tasks outside  the network thread*: It is recommended that delegated tasks indicated by  a handshake status of NEED_TASK are run on a separate thread since they may  block (  http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-22 Thread Sriharsha Chintalapani
Hi Rajini,        Thanks for the details. I did go through your code . There was a discussion before about not having selector related code into the channel or extending the selector it self.  1. *Support for running potentially long-running delegated tasks outside  the network thread*: It is re

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-22 Thread Rajini Sivaram
When we were working on the client-side SSL implementation for Kafka, we found that returning selection interest from handshake() method wasn't sufficient to handle some of the SSL sequences. We resorted to managing the selection key and interest state within SSLChannel to avoid SSL-specific knowle

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-21 Thread Sriharsha Chintalapani
Hi Jay,       Thanks for the review.     1. Isn't the blocking handshake going to be a performance concern? Can we  do the handshake non-blocking instead? If anything that causes connections  to drop can incur blocking network roundtrips won't that eat up all the  network threads immediately? I gu

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-21 Thread Jay Kreps
Hey Sriharsha, Thanks for the excellent write-up. Couple of minor questions: 1. Isn't the blocking handshake going to be a performance concern? Can we do the handshake non-blocking instead? If anything that causes connections to drop can incur blocking network roundtrips won't that eat up all th

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-20 Thread Sriharsha Chintalapani
Hi Jun,            I am using the underlying protocol GSS-API that sasl also uses. I can add details about LDAP/AD . For AD , this is in general the integration of AD to KERBEROS. I.e   kerberos can talk to AD to get the kinit login credentials ( more of a setup details between kerberos and AD)

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-20 Thread Jun Rao
Hi, Harsha, For SASL, a common use case is the integration with LDAP/AD. For completeness, could you describe (or provide a link) how such integration can be done? Also, what about the SSL support, do you plan to describe it in same same KIP or a separate one? Thanks, Jun On Mon, Apr 20, 2015

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-04-20 Thread Sriharsha Chintalapani
Hi,      I updated the KIP-12 with more details. Please take a look   https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809888 Thanks, Harsha On February 11, 2015 at 10:02:43 AM, Harsha (ka...@harsha.io) wrote: Thanks Joe. It will be part of KafkaServer and will run on its own

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Gwen Shapira
Looks good. Thanks for working on this. One note, the Channel implementation from SSL only works on Java7 and up. Since we are still supporting Java 6, I'm working on a lighter wrapper that will be a composite on SocketChannel but will not extend it. Perhaps you'll want to use that. Looking forwa

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Joe Stein
Thanks Harsha, looks good so far. How were you thinking of running the KerberosTicketManager as a standalone process or like controller or is it a layer of code that does the plumbing pieces everywhere? ~ Joestein On Wed, Feb 11, 2015 at 12:18 PM, Harsha wrote: > Hi, > Here is the initial p

[DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Harsha
Hi, Here is the initial proposal for sasl/kerberos implementation for kafka https://cwiki.apache.org/confluence/x/YI4WAw and JIRA https://issues.apache.org/jira/browse/KAFKA-1686. I am currently working on prototype which will add more details to the KIP. Just opening the thread to say the

Fwd: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Harsha
Hi, Here is the initial proposal for sasl/kerberos implementation for kafka https://cwiki.apache.org/confluence/x/YI4WAw and JIRA https://issues.apache.org/jira/browse/KAFKA-1686. I am currently working on prototype which will add more details to the KIP. Just opening the thread to say the