Thanks for clarifying, Parth. I think you are taking the right approach here.
On Fri, Apr 24, 2015 at 11:46 AM, Parth Brahmbhatt <pbrahmbh...@hortonworks.com> wrote: > Sorry Gwen, completely misunderstood the question :-). > > * Does everyone have the privilege to create a new Group and use it to > consume from Topics he's already privileged on? > Yes in current proposal. I did not see an API to create group but if > you > have a READ permission on a TOPIC and WRITE permission on that Group you > are free to join and consume. > > > * Will the CLI tool be used to manage group membership too? > Yes and I think that means I need to add ―group. Updating the KIP. > Thanks > for pointing this out. > > * Groups are kind of ephemeral, right? If all consumers in the group > disconnect the group is gone, AFAIK. Do we preserve the ACLs? Or do we > treat the new group as completely new resource? Can we create ACLs > before the group exists, in anticipation of it getting created? > I have considered any auto delete and auto create as out of scope for > the > first release. So Right now I was going with preserving the acls. Do you > see any issues with this? Auto deleting would mean authorizer will now > have to get into implementation details of kafka which I was trying to > avoid. > > Thanks > Parth > > On 4/24/15, 11:33 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: > >>We are not talking about same Groups :) >> >>I meant, Groups of consumers (which KIP-11 lists as a separate >>resource in the Privilege table) >> >>On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt >><pbrahmbh...@hortonworks.com> wrote: >>> I see Groups as something we can add incrementally in the current model. >>> The acls take principalType: name so groups can be represented as group: >>> groupName. We are not managing group memberships anywhere in kafka and I >>> don’t see the need to do so. >>> >>> So for a topic1 using the CLI an admin can add an acl to grant access to >>> group:kafka-test-users. >>> >>> The authorizer implementation can have a plugin to map authenticated >>>user >>> to groups ( This is how hadoop and storm works). The plugin could be >>> mapping user to linux/ldap/active directory groups but that is again >>>upto >>> the implementation. >>> >>> What we are offering is an interface that is extensible so these >>>features >>> can be added incrementally. I can add support for this in the first >>> release but don’t necessarily see why this would be absolute necessity. >>> >>> Thanks >>> Parth >>> >>> On 4/24/15, 11:00 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >>> >>>>Thanks. >>>> >>>>One more thing I'm missing in the KIP is details on the Group resource >>>>(I think we discussed this and it was just not fully updated): >>>> >>>>* Does everyone have the privilege to create a new Group and use it to >>>>consume from Topics he's already privileged on? >>>>* Will the CLI tool be used to manage group membership too? >>>>* Groups are kind of ephemeral, right? If all consumers in the group >>>>disconnect the group is gone, AFAIK. Do we preserve the ACLs? Or do we >>>>treat the new group as completely new resource? Can we create ACLs >>>>before the group exists, in anticipation of it getting created? >>>> >>>>Its all small details, but it will be difficult to implement KIP-11 >>>>without knowing the answers :) >>>> >>>>Gwen >>>> >>>> >>>>On Fri, Apr 24, 2015 at 9:58 AM, Parth Brahmbhatt >>>><pbrahmbh...@hortonworks.com> wrote: >>>>> You are right, moved it to the default implementation section. >>>>> >>>>> Thanks >>>>> Parth >>>>> >>>>> On 4/24/15, 9:52 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >>>>> >>>>>>Sample ACL JSON and Zookeeper is in public API, but I thought it is >>>>>>part of DefaultAuthorizer (Since Sentry and Argus won't be using >>>>>>Zookeeper). >>>>>>Am I wrong? Or is it the KIP? >>>>>> >>>>>>On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt >>>>>><pbrahmbh...@hortonworks.com> wrote: >>>>>>> Thanks for clarifying Gwen, KIP updated. >>>>>>> >>>>>>> I tried to make the distinction by creating a section for all public >>>>>>>APIs >>>>>>> >>>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizat >>>>>>>io >>>>>>>n+ >>>>>>>In >>>>>>> terface#KIP-11-AuthorizationInterface-PublicInterfacesandclasses >>>>>>> >>>>>>> Let me know if you think there is a better way to reflect this. >>>>>>> >>>>>>> Thanks >>>>>>> Parth >>>>>>> >>>>>>> On 4/24/15, 9:37 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >>>>>>> >>>>>>>>+1 (non-binding) >>>>>>>> >>>>>>>>Two nitpicks for the wiki: >>>>>>>>* Heartbeat is probably a READ and not CLUSTER operation. I'm pretty >>>>>>>>sure new consumers need it to be part of a consumer group. >>>>>>>>* Can you clearly separate which parts are the API (common to every >>>>>>>>Authorizer) and which parts are DefaultAuthorizer implementation? It >>>>>>>>will make reviews and Authorizer implementations a bit easier to >>>>>>>>know >>>>>>>>exactly which is which. >>>>>>>> >>>>>>>>Gwen >>>>>>>> >>>>>>>>On Fri, Apr 24, 2015 at 9:28 AM, Parth Brahmbhatt >>>>>>>><pbrahmbh...@hortonworks.com> wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I would like to open KIP-11 for voting. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Parth >>>>>>>>> >>>>>>>>> On 4/22/15, 1:56 PM, "Parth Brahmbhatt" >>>>>>>>><pbrahmbh...@hortonworks.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>>Hi Jeff, >>>>>>>>>> >>>>>>>>>>Thanks a lot for the review. I think you have a valid point about >>>>>>>>>>acls >>>>>>>>>>being duplicated and the simplest solution would be to modify acls >>>>>>>>>>class >>>>>>>>>>so they hold a set of principals instead of single principal. i.e >>>>>>>>>> >>>>>>>>>><user_a,user_b> has <READ,WRITE,DESCRIBE> Permissions on <Topic1> >>>>>>>>>>from >>>>>>>>>><Host1, Host2, Host3>. >>>>>>>>>> >>>>>>>>>>I think the evaluation order only matters for the permissionType >>>>>>>>>>which >>>>>>>>>>is >>>>>>>>>>Deny acls should be evaluated before allow acls. To give you an >>>>>>>>>>example >>>>>>>>>>suppose we have following acls >>>>>>>>>> >>>>>>>>>>acl1 -> user1 is allowed to READ from all hosts. >>>>>>>>>>acl2 -> host1 is allowed to READ regardless of who is the user. >>>>>>>>>>acl3 -> host2 is allowed to READ regardless of who is the user. >>>>>>>>>> >>>>>>>>>>acl4 -> user1 is denied to READ from host1. >>>>>>>>>> >>>>>>>>>>As stated in the KIP we first evaluate DENY so if user1 tries to >>>>>>>>>>access >>>>>>>>>>from host1 he will be denied(acl4), even though both user1 and >>>>>>>>>>host1 >>>>>>>>>>has >>>>>>>>>>acl’s for allow with wildcards (acl1, acl2). >>>>>>>>>>If user1 tried to READ from host2 , the action will be allowed and >>>>>>>>>>it >>>>>>>>>>does >>>>>>>>>>not matter if we match acl3 or acl1 so I don’t think the >>>>>>>>>>evaluation >>>>>>>>>>order >>>>>>>>>>matters here. >>>>>>>>>> >>>>>>>>>>“Will people actually use hosts with users?” I really don’t know >>>>>>>>>>but >>>>>>>>>>given >>>>>>>>>>ACl’s are part of our Public APIs I thought it is better to try >>>>>>>>>>and >>>>>>>>>>cover >>>>>>>>>>more use cases. If others think this extra complexity is not worth >>>>>>>>>>the >>>>>>>>>>value its adding please raise your concerns so we can discuss if >>>>>>>>>>it >>>>>>>>>>should >>>>>>>>>>be removed from the acl structure. Note that even in absence of >>>>>>>>>>hosts >>>>>>>>>>from >>>>>>>>>>ACL users will still be able to whitelist/blacklist host as long >>>>>>>>>>as >>>>>>>>>>we >>>>>>>>>>start supporting principalType = “host”, easy to add and can be an >>>>>>>>>>incremental improvement. They will however loose the ability to >>>>>>>>>>restrict >>>>>>>>>>access to users just from a set of hosts. >>>>>>>>>> >>>>>>>>>>We agreed to offer a CLI to overcome the JSON acl config >>>>>>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authori >>>>>>>>>>za >>>>>>>>>>ti >>>>>>>>>>on >>>>>>>>>>+I >>>>>>>>>>n >>>>>>>>>>terface#KIP-11-AuthorizationInterface-AclManagement(CLI). I still >>>>>>>>>>like >>>>>>>>>>Jsons but that probably has something to do with me being a >>>>>>>>>>developer >>>>>>>>>>:-). >>>>>>>>>> >>>>>>>>>>Thanks >>>>>>>>>>Parth >>>>>>>>>> >>>>>>>>>>On 4/22/15, 11:38 AM, "Jeff Holoman" <jholo...@cloudera.com> >>>>>>>>>>wrote: >>>>>>>>>> >>>>>>>>>>>Parth, >>>>>>>>>>> >>>>>>>>>>>This is a long thread, so trying to keep up here, sorry if this >>>>>>>>>>>has >>>>>>>>>>>been >>>>>>>>>>>covered before. First, great job on the KIP proposal and work so >>>>>>>>>>>far. >>>>>>>>>>> >>>>>>>>>>>Are we sure that we want to tie host level access to a given >>>>>>>>>>>user? >>>>>>>>>>>My >>>>>>>>>>>understanding is that the ACL will be (omitting some fields) >>>>>>>>>>> >>>>>>>>>>>user_a, host1, host2, host3 >>>>>>>>>>>user_b, host1, host2, host3 >>>>>>>>>>> >>>>>>>>>>>So there would potentially be a lot of redundancy in the configs. >>>>>>>>>>>Does >>>>>>>>>>>it >>>>>>>>>>>make sense to have hosts be at the same level as principal in the >>>>>>>>>>>hierarchy? This way you could just blanket the allowed / denied >>>>>>>>>>>hosts >>>>>>>>>>>and >>>>>>>>>>>only have to worry about the users. So if you follow this, then >>>>>>>>>>> >>>>>>>>>>>we can wildcard the user so we can have a separate list of just >>>>>>>>>>>host-based >>>>>>>>>>>access. What's the order that the perms would be evaluated if a >>>>>>>>>>>there >>>>>>>>>>>was >>>>>>>>>>>more than one match on a principal ? >>>>>>>>>>> >>>>>>>>>>>Is the thought that there wouldn't usually be much overlap on >>>>>>>>>>>hosts? >>>>>>>>>>>I >>>>>>>>>>>guess I can imagine a scenario where I want to offline/online >>>>>>>>>>>access >>>>>>>>>>>to a >>>>>>>>>>>particular hosts or set of hosts and if there was overlap, I'm >>>>>>>>>>>doing >>>>>>>>>>>a >>>>>>>>>>>bunch of alter commands for just a single host. Maybe this is too >>>>>>>>>>>contrived >>>>>>>>>>>an example? >>>>>>>>>>> >>>>>>>>>>>I agree that having this level of granularity gives flexibility >>>>>>>>>>>but I >>>>>>>>>>>wonder if people will actually use it and not just * the hosts >>>>>>>>>>>for >>>>>>>>>>>a >>>>>>>>>>>given >>>>>>>>>>>user and create separate "global" list as i mentioned above? >>>>>>>>>>> >>>>>>>>>>>The only other system I know of that ties users with hosts for >>>>>>>>>>>access >>>>>>>>>>>is >>>>>>>>>>>MySql and I don't love that model. Companies usually standardize >>>>>>>>>>>on >>>>>>>>>>>group >>>>>>>>>>>authorization anyway, are we complicating that issue with the >>>>>>>>>>>inclusion >>>>>>>>>>>of >>>>>>>>>>>hosts attached to users? Additionally I worry about the debt of >>>>>>>>>>>big >>>>>>>>>>>JSON >>>>>>>>>>>configs in the first place, most non-developers find them >>>>>>>>>>>non-intuitive >>>>>>>>>>>already, so anything to ease this I think would be beneficial. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>Thanks >>>>>>>>>>> >>>>>>>>>>>Jeff >>>>>>>>>>> >>>>>>>>>>>On Wed, Apr 22, 2015 at 2:22 PM, Parth Brahmbhatt < >>>>>>>>>>>pbrahmbh...@hortonworks.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Sorry I missed your last questions. I am +0 on adding ―host >>>>>>>>>>>>option >>>>>>>>>>>>for >>>>>>>>>>>> ―list, we could add it for symmetry. Again if this is only a >>>>>>>>>>>>CLI >>>>>>>>>>>>change >>>>>>>>>>>>it >>>>>>>>>>>> can be added later if you mean adding this in authorizer >>>>>>>>>>>>interface >>>>>>>>>>>>then >>>>>>>>>>>>we >>>>>>>>>>>> should make a decision now. >>>>>>>>>>>> >>>>>>>>>>>> Given a choice I would like to actually keep only one option >>>>>>>>>>>>which >>>>>>>>>>>>is >>>>>>>>>>>> resource based get (remove even the get based on principal). I >>>>>>>>>>>>see >>>>>>>>>>>>those >>>>>>>>>>>> (getAcl for principal or host) as special filtering case which >>>>>>>>>>>>can >>>>>>>>>>>>easily >>>>>>>>>>>> be achieved by a third party tool by doing "list all topics" >>>>>>>>>>>>and >>>>>>>>>>>>calling >>>>>>>>>>>> getAcls for each topic and applying filtering logic on that. I >>>>>>>>>>>>really >>>>>>>>>>>> don’t see the need to make those first class citizens of the >>>>>>>>>>>>authorizer >>>>>>>>>>>> interface given these kind of queries will be issued outside of >>>>>>>>>>>>broker >>>>>>>>>>>>JVM >>>>>>>>>>>> so they will not benefit from the caching and because the >>>>>>>>>>>>storage >>>>>>>>>>>>will >>>>>>>>>>>>be >>>>>>>>>>>> indexed on resource both these options even as a first class >>>>>>>>>>>>API >>>>>>>>>>>>will >>>>>>>>>>>>just >>>>>>>>>>>> scan all topic acls and apply filtering logic. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Parth >>>>>>>>>>>> >>>>>>>>>>>> On 4/22/15, 11:08 AM, "Parth Brahmbhatt" >>>>>>>>>>>><pbrahmbh...@hortonworks.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >Please see all the available options here >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Autho >>>>>>>>>>>>ri >>>>>>>>>>>>za >>>>>>>>>>>>ti >>>>>>>>>>>>on >>>>>>>>>>>>+ >>>>>>>>>>>>I >>>>>>>>>>>> >nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I >>>>>>>>>>>>think >>>>>>>>>>>>it >>>>>>>>>>>> >covers both hosts and operations and allows to specify a list >>>>>>>>>>>>for >>>>>>>>>>>>both. >>>>>>>>>>>> > >>>>>>>>>>>> >Thanks >>>>>>>>>>>> >Parth >>>>>>>>>>>> > >>>>>>>>>>>> >From: Tom Graves >>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>> >>>>>>>>>>>> >Reply-To: Tom Graves >>>>>>>>>>>><tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>> >>>>>>>>>>>> >Date: Wednesday, April 22, 2015 at 11:02 AM >>>>>>>>>>>> >To: Parth Brahmbhatt >>>>>>>>>>>> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>, >>>>>>>>>>>> >"dev@kafka.apache.org<mailto:dev@kafka.apache.org>" >>>>>>>>>>>> ><dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >>>>>>>>>>>> >Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka >>>>>>>>>>>>security >>>>>>>>>>>> > >>>>>>>>>>>> >Thanks for the explanations Parth. >>>>>>>>>>>> > >>>>>>>>>>>> >On the configs questions, the way I see it is its more likely >>>>>>>>>>>>to >>>>>>>>>>>> >accidentally give everyone access, especially since you have >>>>>>>>>>>>to >>>>>>>>>>>>run >>>>>>>>>>>>a >>>>>>>>>>>> >separate command to change the acls. If there was some config >>>>>>>>>>>>for >>>>>>>>>>>> >defaults, a cluster admin could change that to be nobody or >>>>>>>>>>>>certain >>>>>>>>>>>>set >>>>>>>>>>>> >of users, then grant others permissions. This would also >>>>>>>>>>>>remove >>>>>>>>>>>>the >>>>>>>>>>>>race >>>>>>>>>>>> >between commands. This is something you can always add later >>>>>>>>>>>>though >>>>>>>>>>>>if >>>>>>>>>>>> >people request it. >>>>>>>>>>>> > >>>>>>>>>>>> >So in kafka-acl.sh how do I actually tell it what the >>>>>>>>>>>>operation >>>>>>>>>>>>is? >>>>>>>>>>>> >kafka-acl.sh --topic testtopic --add --grandprincipal >>>>>>>>>>>>user:joe,user:kate >>>>>>>>>>>> > >>>>>>>>>>>> >where does READ, WRITE, etc go? Can specify as a list so I >>>>>>>>>>>>don't >>>>>>>>>>>>have >>>>>>>>>>>>to >>>>>>>>>>>> >run this a bunch of times for each. >>>>>>>>>>>> > >>>>>>>>>>>> >Do you want to have a --host option for --list so that admins >>>>>>>>>>>>could >>>>>>>>>>>>see >>>>>>>>>>>> >what acls apply to specific host(s)? >>>>>>>>>>>> > >>>>>>>>>>>> >Tom >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt >>>>>>>>>>>> >>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com >>>>>>>>>>>>>>> >>>>>>>>>>>>wrote: >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >FYI, I have modified the KIP to include group as resource. In >>>>>>>>>>>>order >>>>>>>>>>>>to >>>>>>>>>>>> >access “joinGroup” and “commitOFfset” APIs the user will need >>>>>>>>>>>>a >>>>>>>>>>>>read >>>>>>>>>>>> >permission on topic and WRITE permission on group. >>>>>>>>>>>> > >>>>>>>>>>>> >I plan to open a VOTE thread by noon if there are no more >>>>>>>>>>>>concerns. >>>>>>>>>>>> > >>>>>>>>>>>> >Thanks >>>>>>>>>>>> >Parth >>>>>>>>>>>> > >>>>>>>>>>>> >On 4/22/15, 9:03 AM, "Tom Graves" >>>>>>>>>>>> >>>>>>>>>>>>><tgraves...@yahoo.com.INVALID<mailto:tgraves...@yahoo.com.INVAL >>>>>>>>>>>>>ID >>>>>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> > >>>>>>>>>>>> >>Hey everyone, >>>>>>>>>>>> >>Sorry to jump in on the conversation so late. I'm new to >>>>>>>>>>>>Kafka. >>>>>>>>>>>>I'll >>>>>>>>>>>> >>apologize in advance if you have already covered some of my >>>>>>>>>>>>questions. I >>>>>>>>>>>> >>read through the wiki and had some comments and questions. >>>>>>>>>>>> >>1) public enum Operation needs EDIT changed to ALTER >>>>>>>>>>>> > >>>>>>>>>>>> >> Done. >>>>>>>>>>>> > >>>>>>>>>>>> >>2) Does the Authorizer class need a setAcls? Rather then >>>>>>>>>>>>just >>>>>>>>>>>>add >>>>>>>>>>>>to >>>>>>>>>>>>be >>>>>>>>>>>> >>able to set to explicit list and overwrite what was there? I >>>>>>>>>>>>see >>>>>>>>>>>>the >>>>>>>>>>>> >>kafka-acl.sh lists a removeall so I guess you could do >>>>>>>>>>>>removeall >>>>>>>>>>>>and >>>>>>>>>>>>then >>>>>>>>>>>> >>add. I also don't see a removeall in the Authorizer class, >>>>>>>>>>>>is >>>>>>>>>>>>it >>>>>>>>>>>>going >>>>>>>>>>>> >>to loop through them all to remove each one? >>>>>>>>>>>> > >>>>>>>>>>>> > There is an overloaded version of removeAcls in the >>>>>>>>>>>>interface >>>>>>>>>>>>that >>>>>>>>>>>> >takes >>>>>>>>>>>> >in resource as the only input and as described in the javadoc >>>>>>>>>>>>all >>>>>>>>>>>>the >>>>>>>>>>>>acls >>>>>>>>>>>> >attached to that resource will be deleted. To cover the setAcl >>>>>>>>>>>>use >>>>>>>>>>>>case >>>>>>>>>>>> >the caller can first call remove and then add. >>>>>>>>>>>> > >>>>>>>>>>>> >>3) Can someone tell me what the use case to do acls based on >>>>>>>>>>>>the >>>>>>>>>>>>hosts? >>>>>>>>>>>> >>I can see some possibilities just wondering if we can >>>>>>>>>>>>concrete >>>>>>>>>>>>ones >>>>>>>>>>>>where >>>>>>>>>>>> >>one user is allowed from one host but not another. >>>>>>>>>>>> > >>>>>>>>>>>> > I am not sure if I understand the question given the use >>>>>>>>>>>>case >>>>>>>>>>>>you >>>>>>>>>>>> >described in your question is what we are trying to cover with >>>>>>>>>>>>use >>>>>>>>>>>>of >>>>>>>>>>>> >hosts in Acl. There are some additional use cases like “allow >>>>>>>>>>>>access >>>>>>>>>>>>to >>>>>>>>>>>> >any user from host1,host2” but I think primarily it gives the >>>>>>>>>>>>admins >>>>>>>>>>>>the >>>>>>>>>>>> >ability to define acls at a more granular level. >>>>>>>>>>>> > >>>>>>>>>>>> >>4) I'm a bit unclear how the "resource" works in the >>>>>>>>>>>>Authorizer >>>>>>>>>>>>class. >>>>>>>>>>>> >>From what I see we have 2 resources - topics and cluster. >>>>>>>>>>>>If I >>>>>>>>>>>>want >>>>>>>>>>>>to >>>>>>>>>>>> >>add an acl to allow "joe" to CREATE for the cluster then I >>>>>>>>>>>>call >>>>>>>>>>>>addAcls >>>>>>>>>>>> >>with Acl("user: joe", ALLOW, Set(*), Set(CREATE)) and >>>>>>>>>>>>"cluster"? >>>>>>>>>>>>What >>>>>>>>>>>> >>if I want to call addAcls for DESCRIBE on a topic? Is the >>>>>>>>>>>>resource >>>>>>>>>>>>then >>>>>>>>>>>> >>"topic" or is it the topic name? >>>>>>>>>>>> > >>>>>>>>>>>> > We now have 3 resources(added group), please see the >>>>>>>>>>>>updated >>>>>>>>>>>>doc. >>>>>>>>>>>>The >>>>>>>>>>>> >CREATE acl that you described is correct. For any topic >>>>>>>>>>>>operation >>>>>>>>>>>>you >>>>>>>>>>>> >should use topic name as the resource name and for group the >>>>>>>>>>>>user >>>>>>>>>>>>will >>>>>>>>>>>> >provide groupId as resource name. >>>>>>>>>>>> > >>>>>>>>>>>> >>5) reassigning partitions is a CLUSTER_ACTION or superuser? >>>>>>>>>>>>Its >>>>>>>>>>>>not >>>>>>>>>>>> >>totally clear to me the differences between these. what >>>>>>>>>>>>about >>>>>>>>>>>>increasing >>>>>>>>>>>> >># of partitions? >>>>>>>>>>>> > >>>>>>>>>>>> > I see this as an alter topic operation so it is at topic >>>>>>>>>>>>level >>>>>>>>>>>>and >>>>>>>>>>>>the >>>>>>>>>>>> >user must have alter permissions on topic. >>>>>>>>>>>> > >>>>>>>>>>>> >>6) groups are mentioned, are we supporting right away or is >>>>>>>>>>>>that >>>>>>>>>>>>a >>>>>>>>>>>>follow >>>>>>>>>>>> >>on item? (is there going to be a kafka.supergroups) >>>>>>>>>>>> > >>>>>>>>>>>> > I think it can be a separate jira just for braking down >>>>>>>>>>>>the >>>>>>>>>>>>code >>>>>>>>>>>> >review >>>>>>>>>>>> >in smaller chunk. We will support it in first version but I >>>>>>>>>>>>think >>>>>>>>>>>>if >>>>>>>>>>>>we >>>>>>>>>>>> >can not do it for any reason that should not block a release >>>>>>>>>>>>with >>>>>>>>>>>>all >>>>>>>>>>>>the >>>>>>>>>>>> >other authZ work. We made deliberate design choices (like >>>>>>>>>>>>introducing >>>>>>>>>>>>a >>>>>>>>>>>> >principalType in KafkaPrinciapl) to allow supporting groups as >>>>>>>>>>>>an >>>>>>>>>>>> >incremental change. >>>>>>>>>>>> > >>>>>>>>>>>> >>7) Are there config options for setting acls when I create my >>>>>>>>>>>>topic? >>>>>>>>>>>>Or >>>>>>>>>>>> >>do I have to create my topic and then run the kafka-acl.sh >>>>>>>>>>>>script >>>>>>>>>>>>to >>>>>>>>>>>>set >>>>>>>>>>>> >>them? Although its very small, there would be possible race >>>>>>>>>>>>there >>>>>>>>>>>>that >>>>>>>>>>>> >>someone could start producing to topic before acls are set. >>>>>>>>>>>> > >>>>>>>>>>>> > We discussed this yesterday and we agreed to go with >>>>>>>>>>>>kafka-acl.sh. >>>>>>>>>>>>Yes >>>>>>>>>>>> >there is a very very small window of vulnerability but I think >>>>>>>>>>>>that >>>>>>>>>>>>really >>>>>>>>>>>> >does not warrant to change the decision in this case. >>>>>>>>>>>> > >>>>>>>>>>>> >>8) are there configs for cluster level acl defaults? Or does >>>>>>>>>>>>it >>>>>>>>>>>>default >>>>>>>>>>>> >>to superusers on bringing up new cluster and you have to >>>>>>>>>>>>modify >>>>>>>>>>>>with >>>>>>>>>>>>cli. >>>>>>>>>>>> >>thanks,Tom >>>>>>>>>>>> > >>>>>>>>>>>> > No defaults, the default is superusers will have full >>>>>>>>>>>>access. >>>>>>>>>>>>I >>>>>>>>>>>>don’t >>>>>>>>>>>> >think making assumptions about ones security requirement >>>>>>>>>>>>should >>>>>>>>>>>>be >>>>>>>>>>>>our >>>>>>>>>>>> >burden. >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >> >>>>>>>>>>>> >> On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt >>>>>>>>>>>> >>>>>>>>>>>>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.co >>>>>>>>>>>>>>m> >>>>>>>>>>>>>>> >>>>>>>>>>>>wrote: >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> I have added the notes to KIP-11 Open question sections. >>>>>>>>>>>> >> >>>>>>>>>>>> >>Thanks >>>>>>>>>>>> >>Parth >>>>>>>>>>>> >> >>>>>>>>>>>> >>On 4/21/15, 4:49 PM, "Gwen Shapira" >>>>>>>>>>>> >><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> wrote: >>>>>>>>>>>> >> >>>>>>>>>>>> >>>Adding my notes from today's call to the thread: >>>>>>>>>>>> >>> >>>>>>>>>>>> >>>** Deny or Allow all by default? We will add a configuration >>>>>>>>>>>>to >>>>>>>>>>>> >>>control this. The configuration will default to “allow” for >>>>>>>>>>>>backward >>>>>>>>>>>> >>>compatibility. Security admins can set it to "deny" >>>>>>>>>>>> >>> >>>>>>>>>>>> >>>** Storing ACLs for default authorizers: We'll store them in >>>>>>>>>>>>ZK. >>>>>>>>>>>>We'll >>>>>>>>>>>> >>>support pointing the authorizer to any ZK. >>>>>>>>>>>> >>>The use of ZK will be internal to the default authorizer. >>>>>>>>>>>>Authorizer >>>>>>>>>>>> >>>reads ACLs from cache every hour. We proposed having >>>>>>>>>>>>mechanism >>>>>>>>>>>> >>>(possibly via new ZK node) to tell broker to refresh the >>>>>>>>>>>>cache >>>>>>>>>>>> >>>immediately. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>>** Support deny as permission type - we agreed to keep this. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>>** Mapping operations to API: We may need to add Group as a >>>>>>>>>>>>resource, >>>>>>>>>>>> >>>with JoinGroup and OffsetCommit require privilege on the >>>>>>>>>>>>consumer >>>>>>>>>>>> >>>group. >>>>>>>>>>>> >>>This can be something we pass now and authorizers can >>>>>>>>>>>>support >>>>>>>>>>>>in >>>>>>>>>>>> >>>future. - Jay will write specifics to the mailing list >>>>>>>>>>>>discussion. >>>>>>>>>>>> >>> >>>>>>>>>>>> >>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps >>>>>>>>>>>> >>><jay.kr...@gmail.com<mailto:jay.kr...@gmail.com>> wrote: >>>>>>>>>>>> >>>> Following up on the KIP discussion. Two options for >>>>>>>>>>>>authorizing >>>>>>>>>>>> >>>>consumers >>>>>>>>>>>> >>>> to read topic "t" as part of group "g": >>>>>>>>>>>> >>>> 1. READ permission on resource /topic/t >>>>>>>>>>>> >>>> 2. READ permission on resource /topic/t AND WRITE >>>>>>>>>>>>permission >>>>>>>>>>>>on >>>>>>>>>>>> >>>>/group/g >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> The advantage of (1) is that it is simpler. The >>>>>>>>>>>>disadvantage >>>>>>>>>>>>is >>>>>>>>>>>>that >>>>>>>>>>>> >>>>any >>>>>>>>>>>> >>>> member of any group that reads from t can commit offsets >>>>>>>>>>>>as >>>>>>>>>>>>any >>>>>>>>>>>>other >>>>>>>>>>>> >>>> member of a different group. This doesn't effect data >>>>>>>>>>>>security >>>>>>>>>>>>(who >>>>>>>>>>>> >>>>can >>>>>>>>>>>> >>>> access what) but it is a bit of a management issue--a >>>>>>>>>>>>malicious >>>>>>>>>>>>person >>>>>>>>>>>> >>>>can >>>>>>>>>>>> >>>> cause data loss or duplicates for another consumer by >>>>>>>>>>>>committing >>>>>>>>>>>> >>>>offset. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> I think I favor (2) but it's worth it to think it through. >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> -Jay >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt < >>>>>>>>>>>> >>>> >>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>>>>>>>>>> >>>>wrote: >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>>> Hey Jun, >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> Yes and we support wild cards for all acl entities >>>>>>>>>>>>principal, >>>>>>>>>>>>hosts >>>>>>>>>>>> >>>>>and >>>>>>>>>>>> >>>>> operation. >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> Thanks >>>>>>>>>>>> >>>>> Parth >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> On 4/21/15, 9:06 AM, "Jun Rao" >>>>>>>>>>>> >>>>><j...@confluent.io<mailto:j...@confluent.io>> wrote: >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> >Harsha, Parth, >>>>>>>>>>>> >>>>> > >>>>>>>>>>>> >>>>> >Thanks for the clarification. This makes sense. Perhaps >>>>>>>>>>>>we >>>>>>>>>>>>can >>>>>>>>>>>> >>>>>clarify the >>>>>>>>>>>> >>>>> >meaning of those rules in the wiki. >>>>>>>>>>>> >>>>> > >>>>>>>>>>>> >>>>> >Related to this, it seems that we need to support >>>>>>>>>>>>wildcard >>>>>>>>>>>>in >>>>>>>>>>>> >>>>>cli/request >>>>>>>>>>>> >>>>> >protocol for topics? >>>>>>>>>>>> >>>>> > >>>>>>>>>>>> >>>>> >Jun >>>>>>>>>>>> >>>>> > >>>>>>>>>>>> >>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt < >>>>>>>>>>>> >>>>> >>>>>>>>>>>>>pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>wrote: >>>>>>>>>>>> >>>>> > >>>>>>>>>>>> >>>>> >> The iptables on unix supports the DENY operator, not >>>>>>>>>>>>that >>>>>>>>>>>>it >>>>>>>>>>>> >>>>>should >>>>>>>>>>>> >>>>> >> matter. The deny operator can also be used to specify >>>>>>>>>>>>³allow >>>>>>>>>>>>user1 >>>>>>>>>>>> >>>>>to >>>>>>>>>>>> >>>>> >>READ >>>>>>>>>>>> >>>>> >> from topic1 from all hosts but host1,host2². Again we >>>>>>>>>>>>could >>>>>>>>>>>>add a >>>>>>>>>>>> >>>>>host >>>>>>>>>>>> >>>>> >> group semantic and extra complexity around that, not >>>>>>>>>>>>sure >>>>>>>>>>>>if >>>>>>>>>>>>its >>>>>>>>>>>> >>>>>worth >>>>>>>>>>>> >>>>> >>it. >>>>>>>>>>>> >>>>> >> In addition with DENY operator you are now not forced >>>>>>>>>>>>to >>>>>>>>>>>>create a >>>>>>>>>>>> >>>>> >>special >>>>>>>>>>>> >>>>> >> group just to support the authorization use case. I am >>>>>>>>>>>>not >>>>>>>>>>>> >>>>>convinced >>>>>>>>>>>> >>>>> >>that >>>>>>>>>>>> >>>>> >> the operator it self is really all that confusing. >>>>>>>>>>>>There >>>>>>>>>>>>are 3 >>>>>>>>>>>> >>>>>practical >>>>>>>>>>>> >>>>> >> use cases: >>>>>>>>>>>> >>>>> >> - Resource with no acl what so ever -> allow access to >>>>>>>>>>>>everyone ( >>>>>>>>>>>> >>>>>just >>>>>>>>>>>> >>>>> >>for >>>>>>>>>>>> >>>>> >> backward compatibility, I would much rather fail close >>>>>>>>>>>>and >>>>>>>>>>>>force >>>>>>>>>>>> >>>>>users >>>>>>>>>>>> >>>>> >>to >>>>>>>>>>>> >>>>> >> explicitly grant acls that allows access to all >>>>>>>>>>>>users.) >>>>>>>>>>>> >>>>> >> - Resource with some acl attached -> only users that >>>>>>>>>>>>have >>>>>>>>>>>>a >>>>>>>>>>>> >>>>>matching >>>>>>>>>>>> >>>>> >>allow >>>>>>>>>>>> >>>>> >> acl are allowed (i.e. ³allow READ access to topic1 to >>>>>>>>>>>>user1 >>>>>>>>>>>>from >>>>>>>>>>>> >>>>>all >>>>>>>>>>>> >>>>> >> hosts², only user1 has READ access and no other user >>>>>>>>>>>>has >>>>>>>>>>>>access of >>>>>>>>>>>> >>>>>any >>>>>>>>>>>> >>>>> >> kind) >>>>>>>>>>>> >>>>> >> - Resource with some allow and some deny acl attached >>>>>>>>>>>>-> >>>>>>>>>>>>users >>>>>>>>>>>>are >>>>>>>>>>>> >>>>> >>allowed >>>>>>>>>>>> >>>>> >> to perform operation only when they satisfy allow acl >>>>>>>>>>>>and >>>>>>>>>>>>do >>>>>>>>>>>>not >>>>>>>>>>>> >>>>>have >>>>>>>>>>>> >>>>> >> conflicting deny acl. Users that have no acl(allow or >>>>>>>>>>>>deny) >>>>>>>>>>>>will >>>>>>>>>>>> >>>>>still >>>>>>>>>>>> >>>>> >>not >>>>>>>>>>>> >>>>> >> have any access. (i.e. ³allow READ access to topic1 to >>>>>>>>>>>>user1 >>>>>>>>>>>>from >>>>>>>>>>>> >>>>>all >>>>>>>>>>>> >>>>> >> hosts except host1 and host², only user1 has access >>>>>>>>>>>>but >>>>>>>>>>>>not >>>>>>>>>>>>from >>>>>>>>>>>> >>>>>host1 >>>>>>>>>>>> >>>>> >>an >>>>>>>>>>>> >>>>> >> host2) >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >> I think we need to make a decision on deny primarily >>>>>>>>>>>>because >>>>>>>>>>>>with >>>>>>>>>>>> >>>>> >> introduction of acl management API, Acl is now a >>>>>>>>>>>>public >>>>>>>>>>>>class >>>>>>>>>>>>that >>>>>>>>>>>> >>>>>will >>>>>>>>>>>> >>>>> >>be >>>>>>>>>>>> >>>>> >> used by Ranger/Santry and other authroization >>>>>>>>>>>>providers. >>>>>>>>>>>>In >>>>>>>>>>>> >>>>>Current >>>>>>>>>>>> >>>>> >>design >>>>>>>>>>>> >>>>> >> the acl has a permissionType enum field with possible >>>>>>>>>>>>values >>>>>>>>>>>>of >>>>>>>>>>>> >>>>>Allow >>>>>>>>>>>> >>>>> >>and >>>>>>>>>>>> >>>>> >> Deny. If we chose to remove deny we can assume all >>>>>>>>>>>>acls >>>>>>>>>>>>to >>>>>>>>>>>>be >>>>>>>>>>>>of >>>>>>>>>>>> >>>>>allow >>>>>>>>>>>> >>>>> >> type and remove the permissionType field completely. >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >> Thanks >>>>>>>>>>>> >>>>> >> Parth >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira" >>>>>>>>>>>> >>>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> >>>>>>>>>>>>wrote: >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >> >I think thats how its done in pretty much any system >>>>>>>>>>>>I >>>>>>>>>>>>can >>>>>>>>>>>>think >>>>>>>>>>>> >>>>>of. >>>>>>>>>>>> >>>>> >> > >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-- >>>>>>>>>>>Jeff Holoman >>>>>>>>>>>Systems Engineer >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >
