Sorry I must be missing it, that is the doc I'm looking at.
I don't see how you specify the operation?
I don't see a normal --host that goes with list. I see revokehost and
granthost, neither I would think apply to list since --principal. says it
applies to --list.
Copying text here in case I'm looking at the wrong thing:
- kafka-acl.sh --help
A tool to manage acls for kafka cluster and topics. Following are the options
--topic <topicname> name of topic whose acls you want to add/remove.
--cluster this will add/edit cluster level acls where you can control which
users can create topics or list all topics for the cluster or send control
messages to brokers.
--add indicates you are trying to add Acl
--remove indicates you are trying to remove acl
--grantprincipal <principalType: principalName> a comma separated list of
principalType: principalName where currently supported principalType is "user".
When this option is not present but --granthost is specified it will default to
* : * which is wild card that matches all types and all users.
--granthost <host1,host2> a comma separated list of hosts from which the
--grantpricipal is allowed the actions. When this option is not present but
--grantprincipal is specified, it defaults to * which is wildcard matching all
hosts.
--revokeprincipal <principalType: principalName> a comma separated list of
principalType: principalName where currently supported principalType is "user".
When this option is not present but --revokehost is specified , it defaults to
* : * which is wild card that matches all types and all users. Revoke should
only be used to limit the access added by some other grants. A revoke without
an accompanying grant is meaningless as by default only principals in grant
lists are allowed access based on their acls and everyone else is denied.
----revokehost <host1,host2> a comma separated list of hosts from which the
--revokeprincipal will be denied actions. When this option is not present but
--revokeprincipal is specified, it defaults to * which is wildcard matching all
hosts.
--removeAll removes all acls for a topic, only works with topic and not with
cluster.
--list list all acls
--principal <principalType: principalName> Used only with --list to list all
acls for a principal
On Wednesday, April 22, 2015 1:08 PM, Parth Brahmbhatt
<pbrahmbh...@hortonworks.com> wrote:
Please see all the available options here
https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface#KIP-11-AuthorizationInterface-AclManagement(CLI)
. I think it covers both hosts and operations and allows to specify a list for
both.
ThanksParth
From: Tom Graves <tgraves...@yahoo.com>
Reply-To: Tom Graves <tgraves...@yahoo.com>
Date: Wednesday, April 22, 2015 at 11:02 AM
To: Parth Brahmbhatt <pbrahmbh...@hortonworks.com>, "dev@kafka.apache.org"
<dev@kafka.apache.org>
Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security
Thanks for the explanations Parth.
On the configs questions, the way I see it is its more likely to accidentally
give everyone access, especially since you have to run a separate command to
change the acls. If there was some config for defaults, a cluster admin could
change that to be nobody or certain set of users, then grant others
permissions. This would also remove the race between commands. This is
something you can always add later though if people request it.
So in kafka-acl.sh how do I actually tell it what the operation is?kafka-acl.sh
--topic testtopic --add --grandprincipal user:joe,user:kate
where does READ, WRITE, etc go? Can specify as a list so I don't have to run
this a bunch of times for each.
Do you want to have a --host option for --list so that admins could see what
acls apply to specific host(s)?
Tom
On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt
<pbrahmbh...@hortonworks.com> wrote:
FYI, I have modified the KIP to include group as resource. In order to
access “joinGroup” and “commitOFfset” APIs the user will need a read
permission on topic and WRITE permission on group.
I plan to open a VOTE thread by noon if there are no more concerns.
Thanks
Parth
On 4/22/15, 9:03 AM, "Tom Graves" <tgraves...@yahoo.com.INVALID> wrote:
>Hey everyone,
>Sorry to jump in on the conversation so late. I'm new to Kafka. I'll
>apologize in advance if you have already covered some of my questions. I
>read through the wiki and had some comments and questions.
>1) public enum Operation needs EDIT changed to ALTER
> Done.
>2) Does the Authorizer class need a setAcls? Rather then just add to be
>able to set to explicit list and overwrite what was there? I see the
>kafka-acl.sh lists a removeall so I guess you could do removeall and then
>add. I also don't see a removeall in the Authorizer class, is it going
>to loop through them all to remove each one?
There is an overloaded version of removeAcls in the interface that takes
in resource as the only input and as described in the javadoc all the acls
attached to that resource will be deleted. To cover the setAcl use case
the caller can first call remove and then add.
>3) Can someone tell me what the use case to do acls based on the hosts?
>I can see some possibilities just wondering if we can concrete ones where
>one user is allowed from one host but not another.
I am not sure if I understand the question given the use case you
described in your question is what we are trying to cover with use of
hosts in Acl. There are some additional use cases like “allow access to
any user from host1,host2” but I think primarily it gives the admins the
ability to define acls at a more granular level.
>4) I'm a bit unclear how the "resource" works in the Authorizer class.
>From what I see we have 2 resources - topics and cluster. If I want to
>add an acl to allow "joe" to CREATE for the cluster then I call addAcls
>with Acl("user: joe", ALLOW, Set(*), Set(CREATE)) and "cluster"? What
>if I want to call addAcls for DESCRIBE on a topic? Is the resource then
>"topic" or is it the topic name?
We now have 3 resources(added group), please see the updated doc. The
CREATE acl that you described is correct. For any topic operation you
should use topic name as the resource name and for group the user will
provide groupId as resource name.
>5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not
>totally clear to me the differences between these. what about increasing
># of partitions?
I see this as an alter topic operation so it is at topic level and the
user must have alter permissions on topic.
>6) groups are mentioned, are we supporting right away or is that a follow
>on item? (is there going to be a kafka.supergroups)
I think it can be a separate jira just for braking down the code review
in smaller chunk. We will support it in first version but I think if we
can not do it for any reason that should not block a release with all the
other authZ work. We made deliberate design choices (like introducing a
principalType in KafkaPrinciapl) to allow supporting groups as an
incremental change.
>7) Are there config options for setting acls when I create my topic? Or
>do I have to create my topic and then run the kafka-acl.sh script to set
>them? Although its very small, there would be possible race there that
>someone could start producing to topic before acls are set.
We discussed this yesterday and we agreed to go with kafka-acl.sh. Yes
there is a very very small window of vulnerability but I think that really
does not warrant to change the decision in this case.
>8) are there configs for cluster level acl defaults? Or does it default
>to superusers on bringing up new cluster and you have to modify with cli.
>thanks,Tom
No defaults, the default is superusers will have full access. I don’t
think making assumptions about ones security requirement should be our
burden.
>
> On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt
><pbrahmbh...@hortonworks.com> wrote:
>
>
> I have added the notes to KIP-11 Open question sections.
>
>Thanks
>Parth
>
>On 4/21/15, 4:49 PM, "Gwen Shapira" <gshap...@cloudera.com> wrote:
>
>>Adding my notes from today's call to the thread:
>>
>>** Deny or Allow all by default? We will add a configuration to
>>control this. The configuration will default to “allow” for backward
>>compatibility. Security admins can set it to "deny"
>>
>>** Storing ACLs for default authorizers: We'll store them in ZK. We'll
>>support pointing the authorizer to any ZK.
>>The use of ZK will be internal to the default authorizer. Authorizer
>>reads ACLs from cache every hour. We proposed having mechanism
>>(possibly via new ZK node) to tell broker to refresh the cache
>>immediately.
>>
>>** Support deny as permission type - we agreed to keep this.
>>
>>** Mapping operations to API: We may need to add Group as a resource,
>>with JoinGroup and OffsetCommit require privilege on the consumer
>>group.
>>This can be something we pass now and authorizers can support in
>>future. - Jay will write specifics to the mailing list discussion.
>>
>>On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
>>> Following up on the KIP discussion. Two options for authorizing
>>>consumers
>>> to read topic "t" as part of group "g":
>>> 1. READ permission on resource /topic/t
>>> 2. READ permission on resource /topic/t AND WRITE permission on
>>>/group/g
>>>
>>> The advantage of (1) is that it is simpler. The disadvantage is that
>>>any
>>> member of any group that reads from t can commit offsets as any other
>>> member of a different group. This doesn't effect data security (who can
>>> access what) but it is a bit of a management issue--a malicious person
>>>can
>>> cause data loss or duplicates for another consumer by committing
>>>offset.
>>>
>>> I think I favor (2) but it's worth it to think it through.
>>>
>>> -Jay
>>>
>>> On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt <
>>> pbrahmbh...@hortonworks.com> wrote:
>>>
>>>> Hey Jun,
>>>>
>>>> Yes and we support wild cards for all acl entities principal, hosts
>>>>and
>>>> operation.
>>>>
>>>> Thanks
>>>> Parth
>>>>
>>>> On 4/21/15, 9:06 AM, "Jun Rao" <j...@confluent.io> wrote:
>>>>
>>>> >Harsha, Parth,
>>>> >
>>>> >Thanks for the clarification. This makes sense. Perhaps we can
>>>>clarify the
>>>> >meaning of those rules in the wiki.
>>>> >
>>>> >Related to this, it seems that we need to support wildcard in
>>>>cli/request
>>>> >protocol for topics?
>>>> >
>>>> >Jun
>>>> >
>>>> >On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt <
>>>> >pbrahmbh...@hortonworks.com> wrote:
>>>> >
>>>> >> The iptables on unix supports the DENY operator, not that it should
>>>> >> matter. The deny operator can also be used to specify ³allow user1
>>>>to
>>>> >>READ
>>>> >> from topic1 from all hosts but host1,host2². Again we could add a
>>>>host
>>>> >> group semantic and extra complexity around that, not sure if its
>>>>worth
>>>> >>it.
>>>> >> In addition with DENY operator you are now not forced to create a
>>>> >>special
>>>> >> group just to support the authorization use case. I am not
>>>>convinced
>>>> >>that
>>>> >> the operator it self is really all that confusing. There are 3
>>>>practical
>>>> >> use cases:
>>>> >> - Resource with no acl what so ever -> allow access to everyone (
>>>>just
>>>> >>for
>>>> >> backward compatibility, I would much rather fail close and force
>>>>users
>>>> >>to
>>>> >> explicitly grant acls that allows access to all users.)
>>>> >> - Resource with some acl attached -> only users that have a
>>>>matching
>>>> >>allow
>>>> >> acl are allowed (i.e. ³allow READ access to topic1 to user1 from
>>>>all
>>>> >> hosts², only user1 has READ access and no other user has access of
>>>>any
>>>> >> kind)
>>>> >> - Resource with some allow and some deny acl attached -> users are
>>>> >>allowed
>>>> >> to perform operation only when they satisfy allow acl and do not
>>>>have
>>>> >> conflicting deny acl. Users that have no acl(allow or deny) will
>>>>still
>>>> >>not
>>>> >> have any access. (i.e. ³allow READ access to topic1 to user1 from
>>>>all
>>>> >> hosts except host1 and host², only user1 has access but not from
>>>>host1
>>>> >>an
>>>> >> host2)
>>>> >>
>>>> >> I think we need to make a decision on deny primarily because with
>>>> >> introduction of acl management API, Acl is now a public class that
>>>>will
>>>> >>be
>>>> >> used by Ranger/Santry and other authroization providers. In Current
>>>> >>design
>>>> >> the acl has a permissionType enum field with possible values of
>>>>Allow
>>>> >>and
>>>> >> Deny. If we chose to remove deny we can assume all acls to be of
>>>>allow
>>>> >> type and remove the permissionType field completely.
>>>> >>
>>>> >> Thanks
>>>> >> Parth
>>>> >>
>>>> >> On 4/20/15, 6:12 PM, "Gwen Shapira" <gshap...@cloudera.com> wrote:
>>>> >>
>>>> >> >I think thats how its done in pretty much any system I can think
>>>>of.
>>>> >> >
>>>> >>
>>>> >>
>>>>
>>>>
>
>
>
>
