Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Gwen Shapira
The KIP and design were accepted, so the WIKI should say "accepted" or something similar. Specific patch status is reflected in the JIRA. On Thu, May 21, 2015 at 8:37 PM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > I am sorry to be ignorant about this but what is the new state? Adopt

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Parth Brahmbhatt
I am sorry to be ignorant about this but what is the new state? Adopted seems too early given we are still in code review process. Should I just make it ³Code review²? Thanks Parth On 5/21/15, 8:43 AM, "Jun Rao" wrote: >Parth, > >Thanks for driving this. Could you update the status of the KIP i

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Jun Rao
Parth, Thanks for driving this. Could you update the status of the KIP in the wiki? Thanks, Jun On Wed, May 20, 2015 at 2:37 PM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > This vote is now Closed with 4 binding +1s and 4 non binding +1s. > > Thanks > Parth > > On 5/20/15, 12:04 P

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-20 Thread Parth Brahmbhatt
This vote is now Closed with 4 binding +1s and 4 non binding +1s. Thanks Parth On 5/20/15, 12:04 PM, "Joel Koshy" wrote: >+1 > >On Fri, May 15, 2015 at 04:18:49PM +, Parth Brahmbhatt wrote: >> Hi, >> >> Opening the voting thread for KIP-11. >> >> Link to the KIP: >>https://cwiki.apache.or

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-20 Thread Joel Koshy
+1 On Fri, May 15, 2015 at 04:18:49PM +, Parth Brahmbhatt wrote: > Hi, > > Opening the voting thread for KIP-11. > > Link to the KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 > > Th

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-18 Thread Joe Stein
+1 ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Fri, May 15, 2015 at 7:35 PM, Jun Rao wrote: > +1 > > Thanks, > > Jun > > On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt < > pbrahmbh...@hortonworks.com> wrote: > > > Hi, > > > > Op

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Jun Rao
+1 Thanks, Jun On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > Hi, > > Opening the voting thread for KIP-11. > > Link to the KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > Link to Jira: https://issues.apache.

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Jay Kreps
+1 -Jay On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > Hi, > > Opening the voting thread for KIP-11. > > Link to the KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > Link to Jira: https://issues.apache.org/jira

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Tom Graves
+1 non-binding. Tom Graves On Friday, May 15, 2015 2:00 PM, Don Bosco Durai wrote: +1 non-binding On 5/15/15, 11:43 AM, "Gwen Shapira" wrote: >+1 non-binding > >On Fri, May 15, 2015 at 9:12 PM, Harsha wrote: > >> +1 non-binding >> >> >> >> >> >> >> On Fri, May 15, 2015 at 9:18 A

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Don Bosco Durai
+1 non-binding On 5/15/15, 11:43 AM, "Gwen Shapira" wrote: >+1 non-binding > >On Fri, May 15, 2015 at 9:12 PM, Harsha wrote: > >> +1 non-binding >> >> >> >> >> >> >> On Fri, May 15, 2015 at 9:18 AM -0700, "Parth Brahmbhatt" < >> pbrahmbh...@hortonworks.com> wrote: >> >> >> >> >> >> >> >> >> >>

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Gwen Shapira
+1 non-binding On Fri, May 15, 2015 at 9:12 PM, Harsha wrote: > +1 non-binding > > > > > > > On Fri, May 15, 2015 at 9:18 AM -0700, "Parth Brahmbhatt" < > pbrahmbh...@hortonworks.com> wrote: > > > > > > > > > > > Hi, > > Opening the voting thread for KIP-11. > > Link to the KIP: > https://cwiki.

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Harsha
+1 non-binding On Fri, May 15, 2015 at 9:18 AM -0700, "Parth Brahmbhatt" wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira: https://issues.apache.org/jira/browse/KAFK

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-05-01 Thread Jun Rao
nt from phone > > _ > From: Gwen Shapira mailto:gshap...@cloudera.com>> > Sent: Thursday, April 30, 2015 5:32 PM > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security > To: mailto:dev@kafka.apache.org>> > > > On Thu, Apr 30

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Don Bosco Durai
so it can keep moving forward. >> > >> >> > >> >> > >> ~ Joe Stein >> > >> >> > >> On Tue, Apr 28, 2015 at 3:33 AM, Sun, Dapeng >> > >>wrote: >> > >> >> > >> > Thank you for

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
http://docs.aws.amazon.com/kinesis/latest/APIReference/CommonErrors.html From: Gwen Shapira Sent: Thursday, April 30, 2015 6:05 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security I think Kafka's behavior should be

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
> > Sent from phone > > _ > From: Gwen Shapira mailto:gshap...@cloudera.com>> > Sent: Thursday, April 30, 2015 5:32 PM > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security > To: mailto:dev@kafka.apache.org>> > > > On T

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Don Bosco Durai
ting the security concern. System must be ensure disallowing >> the access by implementing the security correctly. Not based on >>security by >> obscurity. >> >> Regards, >> Suresh >> >> Sent from phone >> >> _ >> F

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
- - - > > > >On Thu, Apr 30, 2015 at 6:59 PM, Suresh Srinivas > >wrote: > > > >> Joe, > >> > >> Can you add more details on what generalization looks like? Also is > >>this a > >> design issue or code issue? > >> > &g

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security I kind of thought of the authorization module as something that happens in handle(request: RequestChannel.Reuqest) in the request.requestId match If the request doesn't do what it is allowed too it should stop

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
> creation, deletion, access etc.? > > Regards, > Suresh > > Sent from phone > > _ > From: Joe Stein mailto:joe.st...@stealth.ly>> > Sent: Thursday, April 30, 2015 3:27 PM > Subject: Re: [VOTE] KIP-11- Authorization design for kafka secu

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
from phone > > _ > From: Gwen Shapira mailto:gshap...@cloudera.com>> > Sent: Thursday, April 30, 2015 10:14 AM > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security > To: mailto:dev@kafka.apache.org>> > > > * Regarding add

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
_____ > From: Gwen Shapira mailto:gshap...@cloudera.com>> > Sent: Thursday, April 30, 2015 10:14 AM > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security > To: mailto:dev@kafka.apache.org>> > > > * Regarding additional authorizers: > Prasad

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
phone _ From: Gwen Shapira mailto:gshap...@cloudera.com>> Sent: Thursday, April 30, 2015 10:14 AM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: mailto:dev@kafka.apache.org>> * Regarding additional authorizers: Prasad, who is a PMC on Apache Sentry review

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
>> >> > >>this > >> >> > >> wasbrought up already or I missed it. > >> >> > >> > >> >> > >> I read through the KIP and the thread(s) and a couple of things > >> >>jumped > >> >> > >>out. >

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Parth Brahmbhatt
how we can >> >>know >> >> > >>the >> >> > >>code works. >> >> > >> >> >> > >> >> >> > >> >> >> > >>- We need some implementation/example/sample

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Parth Brahmbhatt
We currently don't have any mechanism for specifying IP >>>ranges >>> (or >>> > >> host >>> > >> > >ranges) at all. I think its a pretty significant deficiency, >>>but it >>> > >>does >>> > >>

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
> > Thank you for your rapid reply, Parth. > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > >* I think the wiki already describes the precedence order as > >>Deny > >> > >> &g

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Sriharsha Chintalapani
; > >> > >2. We currently don't have any mechanism for specifying IP ranges > (or > > >> host > > >> > >ranges) at all. I think its a pretty significant deficiency, but it > > >>does > > >> > mean that we d

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Parth Brahmbhatt
gt; > >We have a call tomorrow (Tuesday, April 28) at 3pm PST - to >>discuss >> > >>this >> > >> > and other outstanding design issues (not all related to >>security). >> If >> > >>you >> > >

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
d up being less secure. The rule "Deny always wins" is very easy > to > > >> grasp. > > >> > Yes, I'm agreed with your point: we should not make the rule > complex. > > >> > > > >> > >2. We currently don't have any mech

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
sue of blocking a large > >> range > >> > while unblocking few servers in the range. > >> > Support ranges sounds reasonable. If this feature will be in > >>development > >> > plan, I also don't think we can put "the best matching acl"

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Parth Brahmbhatt
about the issue of blocking a large >>> range >>> > while unblocking few servers in the range. >>> > Support ranges sounds reasonable. If this feature will be in >>>development >>> > plan, I also don't think we can put "the best mat

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Parth Brahmbhatt
We have a call tomorrow (Tuesday, April 28) at 3pm PST - to discuss >>this >> > and other outstanding design issues (not all related to security). If >>you >> > are interested in joining - let me know and I'll forward you the >>invite. >> > Thank yo

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Jun Rao
t; > Thank you, Gwen. I have the invite and I should be at home at that time. > > But due to network issue, I may can't join the meeting smoothly. > > > > Regards > > Dapeng > > > > -Original Message- > > From: Gwen Shapira [mailto:gshap

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
> Regards > Dapeng > > -----Original Message- > From: Gwen Shapira [mailto:gshap...@cloudera.com] > Sent: Tuesday, April 28, 2015 1:31 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-11- Authorization design for kafka security > > While I see the advantag

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-28 Thread Sun, Dapeng
: Gwen Shapira [mailto:gshap...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something like: "deny user X from hosts h1...h200" also "

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Gwen Shapira
5 PM, Sun, Dapeng wrote: > Attach the image. > > https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-acl1.png > > Regards > Dapeng > > From: Sun, Dapeng [mailto:dapeng@intel.com] > Sent: Tuesday, April 28, 2015 11:44 AM > To: dev@kafka.apache.org > Subje

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Sun, Dapeng
Attach the image. https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-acl1.png Regards Dapeng From: Sun, Dapeng [mailto:dapeng@intel.com] Sent: Tuesday, April 28, 2015 11:44 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-11- Authorization design for kafka security

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
* We are not supporting regex matching to any of the strings (host,resource,principal) yet but this can be added. We have a special wild card (*) to refer to ALL but there is no other regex matching going on right now. We can associate CREATE with topics as you are proposing once KIP-4 is merged I

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Jun Rao
Parth, I was thinking that in a multi-tenant environment, an admin may want to carve out some topic space to a user. For example, allow user X to create any topic of X_*. Not sure how critical it is though. Also, with the current api, what would the admin do to replicate the acls from one cluster

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
ration and one host. So we could make sure there >are not many acls with same meaning and make acl management easily. > > >Regards >Dapeng > >-Original Message- >From: Jun Rao [mailto:j...@confluent.io] >Sent: Monday, April 27, 2015 5:02 AM >To: dev@kafka.apache.

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
Thanks for your comments Jun. * Renamed the resource to consumer-group in wiki. * I don’t see a use case where admins/users would want to reserve topic names in advance. Can you describe why this would be needed. Thanks Parth On 4/26/15, 2:01 PM, "Jun Rao" wrote: >A few more minor comments. >

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-26 Thread Sun, Dapeng
okeeper or memory we may better to separate to one-principle, one-operation and one host. So we could make sure there are not many acls with same meaning and make acl management easily. Regards Dapeng -Original Message----- From: Jun Rao [mailto:j...@confluent.io] Sent: Monday, April 27, 2

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-26 Thread Jun Rao
A few more minor comments. 100. To make it clear, perhaps we should rename the resource "group" to consumer-group. We can probably make the same change in CLI as well so that it's not confused with user group. 101. Currently, create is only at the cluster level. Should it also be at topic level?

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-25 Thread Gwen Shapira
Thanks for clarifying, Parth. I think you are taking the right approach here. On Fri, Apr 24, 2015 at 11:46 AM, Parth Brahmbhatt wrote: > Sorry Gwen, completely misunderstood the question :-). > > * Does everyone have the privilege to create a new Group and use it to > consume from Topics he's al

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
I will move the comments about subject versus principal wrt session to the PR above. The comments around keys, etc are more appropriate there. If I tie this together with my comments in the thread on SASL / Kerberos, what I am having a hard time figuring out are the pluggable framework for both a

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Sorry Gwen, completely misunderstood the question :-). * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? Yes in current proposal. I did not see an API to create group but if you have a READ permission on a TOPIC and WRITE

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sorry, for the confusion. I'm not sure my last email is clear enough either... Consumers will have a Principal which may belong to a group. But consumer configuration also have a group.id, which controls how partitions are shared between consumers and how offsets are committed. I'm talking about t

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
We are not talking about same Groups :) I meant, Groups of consumers (which KIP-11 lists as a separate resource in the Privilege table) On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt wrote: > I see Groups as something we can add incrementally in the current model. > The acls take principalTy

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
I see Groups as something we can add incrementally in the current model. The acls take principalType: name so groups can be represented as group: groupName. We are not managing group memberships anywhere in kafka and I don’t see the need to do so. So for a topic1 using the CLI an admin can add an

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for your comments Gari. My responses are inline. Thanks Parth On 4/24/15, 10:36 AM, "Gari Singh" wrote: >Sorry - fat fingered send ... > > >Not sure if my "newbie" vote will count, but I think you are getting >pretty >close here. > >Couple of things: > >1) I know the Session object is fr

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Thanks. One more thing I'm missing in the KIP is details on the Group resource (I think we discussed this and it was just not fully updated): * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? * Will the CLI tool be used to manag

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Sorry - fat fingered send ... Not sure if my "newbie" vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Not sure if my "newbie" vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this is because a Subject can have m

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Sriharsha Chintalapani
+1 (non-binding) --  Harsha On April 24, 2015 at 9:59:09 AM, Parth Brahmbhatt (pbrahmbh...@hortonworks.com) wrote: You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, "Gwen Shapira" wrote: >Sample ACL JSON and Zookeeper is in public API,

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, "Gwen Shapira" wrote: >Sample ACL JSON and Zookeeper is in public API, but I thought it is >part of DefaultAuthorizer (Since Sentry and Argus won't be using >Zookeeper). >Am I wrong? Or is it the KI

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sample ACL JSON and Zookeeper is in public API, but I thought it is part of DefaultAuthorizer (Since Sentry and Argus won't be using Zookeeper). Am I wrong? Or is it the KIP? On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt wrote: > Thanks for clarifying Gwen, KIP updated. > > I tried to make th

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for clarifying Gwen, KIP updated. I tried to make the distinction by creating a section for all public APIs https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+In terface#KIP-11-AuthorizationInterface-PublicInterfacesandclasses Let me know if you think there is a bette

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
+1 (non-binding) Two nitpicks for the wiki: * Heartbeat is probably a READ and not CLUSTER operation. I'm pretty sure new consumers need it to be part of a consumer group. * Can you clearly separate which parts are the API (common to every Authorizer) and which parts are DefaultAuthorizer implemen

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Hi, I would like to open KIP-11 for voting. Thanks Parth On 4/22/15, 1:56 PM, "Parth Brahmbhatt" wrote: >Hi Jeff, > >Thanks a lot for the review. I think you have a valid point about acls >being duplicated and the simplest solution would be to modify acls class >so they hold a set of principal