Thanks! Makes sense.
Tom
On Wednesday, April 22, 2015 1:25 PM, Parth Brahmbhatt
<pbrahmbh...@hortonworks.com> wrote:
You are right , I forgot to mention the ―operation option in CLI , I just
added it. Sorry for about the confusion.
Thanks
Parth
On 4/22/15, 11:22 AM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com>
wrote:
>Sorry I missed your last questions. I am +0 on adding ―host option for
>―list, we could add it for symmetry. Again if this is only a CLI change it
>can be added later if you mean adding this in authorizer interface then we
>should make a decision now.
>
>Given a choice I would like to actually keep only one option which is
>resource based get (remove even the get based on principal). I see those
>(getAcl for principal or host) as special filtering case which can easily
>be achieved by a third party tool by doing "list all topics" and calling
>getAcls for each topic and applying filtering logic on that. I really
>don’t see the need to make those first class citizens of the authorizer
>interface given these kind of queries will be issued outside of broker JVM
>so they will not benefit from the caching and because the storage will be
>indexed on resource both these options even as a first class API will just
>scan all topic acls and apply filtering logic.
>
>Thanks
>Parth
>
>On 4/22/15, 11:08 AM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com>
>wrote:
>
>>Please see all the available options here
>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+
>>I
>>nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it
>>covers both hosts and operations and allows to specify a list for both.
>>
>>Thanks
>>Parth
>>
>>From: Tom Graves <tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>>
>>Reply-To: Tom Graves <tgraves...@yahoo.com<mailto:tgraves...@yahoo.com>>
>>Date: Wednesday, April 22, 2015 at 11:02 AM
>>To: Parth Brahmbhatt
>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>>,
>>"dev@kafka.apache.org<mailto:dev@kafka.apache.org>"
>><dev@kafka.apache.org<mailto:dev@kafka.apache.org>>
>>Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security
>>
>>Thanks for the explanations Parth.
>>
>>On the configs questions, the way I see it is its more likely to
>>accidentally give everyone access, especially since you have to run a
>>separate command to change the acls. If there was some config for
>>defaults, a cluster admin could change that to be nobody or certain set
>>of users, then grant others permissions. This would also remove the race
>>between commands. This is something you can always add later though if
>>people request it.
>>
>>So in kafka-acl.sh how do I actually tell it what the operation is?
>>kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate
>>
>>where does READ, WRITE, etc go? Can specify as a list so I don't have to
>>run this a bunch of times for each.
>>
>>Do you want to have a --host option for --list so that admins could see
>>what acls apply to specific host(s)?
>>
>>Tom
>>
>>
>>
>>On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt
>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> wrote:
>>
>>
>>
>>FYI, I have modified the KIP to include group as resource. In order to
>>access “joinGroup” and “commitOFfset” APIs the user will need a read
>>permission on topic and WRITE permission on group.
>>
>>I plan to open a VOTE thread by noon if there are no more concerns.
>>
>>Thanks
>>Parth
>>
>>On 4/22/15, 9:03 AM, "Tom Graves"
>><tgraves...@yahoo.com.INVALID<mailto:tgraves...@yahoo.com.INVALID>>
>>wrote:
>>
>>>Hey everyone,
>>>Sorry to jump in on the conversation so late. I'm new to Kafka. I'll
>>>apologize in advance if you have already covered some of my questions.
>>>I
>>>read through the wiki and had some comments and questions.
>>>1) public enum Operation needs EDIT changed to ALTER
>>
>>> Done.
>>
>>>2) Does the Authorizer class need a setAcls? Rather then just add to be
>>>able to set to explicit list and overwrite what was there? I see the
>>>kafka-acl.sh lists a removeall so I guess you could do removeall and
>>>then
>>>add. I also don't see a removeall in the Authorizer class, is it going
>>>to loop through them all to remove each one?
>>
>> There is an overloaded version of removeAcls in the interface that
>>takes
>>in resource as the only input and as described in the javadoc all the
>>acls
>>attached to that resource will be deleted. To cover the setAcl use case
>>the caller can first call remove and then add.
>>
>>>3) Can someone tell me what the use case to do acls based on the hosts?
>>>I can see some possibilities just wondering if we can concrete ones
>>>where
>>>one user is allowed from one host but not another.
>>
>> I am not sure if I understand the question given the use case you
>>described in your question is what we are trying to cover with use of
>>hosts in Acl. There are some additional use cases like “allow access to
>>any user from host1,host2” but I think primarily it gives the admins the
>>ability to define acls at a more granular level.
>>
>>>4) I'm a bit unclear how the "resource" works in the Authorizer class.
>>>From what I see we have 2 resources - topics and cluster. If I want to
>>>add an acl to allow "joe" to CREATE for the cluster then I call addAcls
>>>with Acl("user: joe", ALLOW, Set(*), Set(CREATE)) and "cluster"? What
>>>if I want to call addAcls for DESCRIBE on a topic? Is the resource then
>>>"topic" or is it the topic name?
>>
>> We now have 3 resources(added group), please see the updated doc. The
>>CREATE acl that you described is correct. For any topic operation you
>>should use topic name as the resource name and for group the user will
>>provide groupId as resource name.
>>
>>>5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not
>>>totally clear to me the differences between these. what about
>>>increasing
>>># of partitions?
>>
>> I see this as an alter topic operation so it is at topic level and
>>the
>>user must have alter permissions on topic.
>>
>>>6) groups are mentioned, are we supporting right away or is that a
>>>follow
>>>on item? (is there going to be a kafka.supergroups)
>>
>> I think it can be a separate jira just for braking down the code
>>review
>>in smaller chunk. We will support it in first version but I think if we
>>can not do it for any reason that should not block a release with all the
>>other authZ work. We made deliberate design choices (like introducing a
>>principalType in KafkaPrinciapl) to allow supporting groups as an
>>incremental change.
>>
>>>7) Are there config options for setting acls when I create my topic? Or
>>>do I have to create my topic and then run the kafka-acl.sh script to set
>>>them? Although its very small, there would be possible race there that
>>>someone could start producing to topic before acls are set.
>>
>> We discussed this yesterday and we agreed to go with kafka-acl.sh.
>>Yes
>>there is a very very small window of vulnerability but I think that
>>really
>>does not warrant to change the decision in this case.
>>
>>>8) are there configs for cluster level acl defaults? Or does it default
>>>to superusers on bringing up new cluster and you have to modify with
>>>cli.
>>>thanks,Tom
>>
>> No defaults, the default is superusers will have full access. I don’t
>>think making assumptions about ones security requirement should be our
>>burden.
>>
>>
>>>
>>> On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt
>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> wrote:
>>>
>>>
>>> I have added the notes to KIP-11 Open question sections.
>>>
>>>Thanks
>>>Parth
>>>
>>>On 4/21/15, 4:49 PM, "Gwen Shapira"
>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> wrote:
>>>
>>>>Adding my notes from today's call to the thread:
>>>>
>>>>** Deny or Allow all by default? We will add a configuration to
>>>>control this. The configuration will default to “allow” for backward
>>>>compatibility. Security admins can set it to "deny"
>>>>
>>>>** Storing ACLs for default authorizers: We'll store them in ZK. We'll
>>>>support pointing the authorizer to any ZK.
>>>>The use of ZK will be internal to the default authorizer. Authorizer
>>>>reads ACLs from cache every hour. We proposed having mechanism
>>>>(possibly via new ZK node) to tell broker to refresh the cache
>>>>immediately.
>>>>
>>>>** Support deny as permission type - we agreed to keep this.
>>>>
>>>>** Mapping operations to API: We may need to add Group as a resource,
>>>>with JoinGroup and OffsetCommit require privilege on the consumer
>>>>group.
>>>>This can be something we pass now and authorizers can support in
>>>>future. - Jay will write specifics to the mailing list discussion.
>>>>
>>>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps
>>>><jay.kr...@gmail.com<mailto:jay.kr...@gmail.com>> wrote:
>>>>> Following up on the KIP discussion. Two options for authorizing
>>>>>consumers
>>>>> to read topic "t" as part of group "g":
>>>>> 1. READ permission on resource /topic/t
>>>>> 2. READ permission on resource /topic/t AND WRITE permission on
>>>>>/group/g
>>>>>
>>>>> The advantage of (1) is that it is simpler. The disadvantage is that
>>>>>any
>>>>> member of any group that reads from t can commit offsets as any other
>>>>> member of a different group. This doesn't effect data security (who
>>>>>can
>>>>> access what) but it is a bit of a management issue--a malicious
>>>>>person
>>>>>can
>>>>> cause data loss or duplicates for another consumer by committing
>>>>>offset.
>>>>>
>>>>> I think I favor (2) but it's worth it to think it through.
>>>>>
>>>>> -Jay
>>>>>
>>>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt <
>>>>> pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>>
>>>>>wrote:
>>>>>
>>>>>> Hey Jun,
>>>>>>
>>>>>> Yes and we support wild cards for all acl entities principal, hosts
>>>>>>and
>>>>>> operation.
>>>>>>
>>>>>> Thanks
>>>>>> Parth
>>>>>>
>>>>>> On 4/21/15, 9:06 AM, "Jun Rao"
>>>>>><j...@confluent.io<mailto:j...@confluent.io>> wrote:
>>>>>>
>>>>>> >Harsha, Parth,
>>>>>> >
>>>>>> >Thanks for the clarification. This makes sense. Perhaps we can
>>>>>>clarify the
>>>>>> >meaning of those rules in the wiki.
>>>>>> >
>>>>>> >Related to this, it seems that we need to support wildcard in
>>>>>>cli/request
>>>>>> >protocol for topics?
>>>>>> >
>>>>>> >Jun
>>>>>> >
>>>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt <
>>>>>> >pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>>
>>>>>>wrote:
>>>>>> >
>>>>>> >> The iptables on unix supports the DENY operator, not that it
>>>>>>should
>>>>>> >> matter. The deny operator can also be used to specify ³allow
>>>>>>user1
>>>>>>to
>>>>>> >>READ
>>>>>> >> from topic1 from all hosts but host1,host2². Again we could add a
>>>>>>host
>>>>>> >> group semantic and extra complexity around that, not sure if its
>>>>>>worth
>>>>>> >>it.
>>>>>> >> In addition with DENY operator you are now not forced to create a
>>>>>> >>special
>>>>>> >> group just to support the authorization use case. I am not
>>>>>>convinced
>>>>>> >>that
>>>>>> >> the operator it self is really all that confusing. There are 3
>>>>>>practical
>>>>>> >> use cases:
>>>>>> >> - Resource with no acl what so ever -> allow access to everyone (
>>>>>>just
>>>>>> >>for
>>>>>> >> backward compatibility, I would much rather fail close and force
>>>>>>users
>>>>>> >>to
>>>>>> >> explicitly grant acls that allows access to all users.)
>>>>>> >> - Resource with some acl attached -> only users that have a
>>>>>>matching
>>>>>> >>allow
>>>>>> >> acl are allowed (i.e. ³allow READ access to topic1 to user1 from
>>>>>>all
>>>>>> >> hosts², only user1 has READ access and no other user has access
>>>>>>of
>>>>>>any
>>>>>> >> kind)
>>>>>> >> - Resource with some allow and some deny acl attached -> users
>>>>>>are
>>>>>> >>allowed
>>>>>> >> to perform operation only when they satisfy allow acl and do not
>>>>>>have
>>>>>> >> conflicting deny acl. Users that have no acl(allow or deny) will
>>>>>>still
>>>>>> >>not
>>>>>> >> have any access. (i.e. ³allow READ access to topic1 to user1 from
>>>>>>all
>>>>>> >> hosts except host1 and host², only user1 has access but not from
>>>>>>host1
>>>>>> >>an
>>>>>> >> host2)
>>>>>> >>
>>>>>> >> I think we need to make a decision on deny primarily because with
>>>>>> >> introduction of acl management API, Acl is now a public class
>>>>>>that
>>>>>>will
>>>>>> >>be
>>>>>> >> used by Ranger/Santry and other authroization providers. In
>>>>>>Current
>>>>>> >>design
>>>>>> >> the acl has a permissionType enum field with possible values of
>>>>>>Allow
>>>>>> >>and
>>>>>> >> Deny. If we chose to remove deny we can assume all acls to be of
>>>>>>allow
>>>>>> >> type and remove the permissionType field completely.
>>>>>> >>
>>>>>> >> Thanks
>>>>>> >> Parth
>>>>>> >>
>>>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira"
>>>>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> wrote:
>>>>>> >>
>>>>>> >> >I think thats how its done in pretty much any system I can think
>>>>>>of.
>>>>>> >> >
>>>>>> >>
>>>>>> >>
>>>>>>
>>>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
