According to the pseudo code, if you have a rule "deny user1", then it
essentially denies all users?
Thanks,
Jun
On Mon, Apr 20, 2015 at 5:16 PM, Parth Brahmbhatt <
pbrahmbh...@hortonworks.com> wrote:
>
> Here is a pseudo code that explains my current approach:
>
> acls = authorizer.getAcl(resource)
> if(acls == null || acls.isEmpty) {
> allow all requests for backward compatibility. (any topics that
> were
> created prior to security support will not have acls) This is debatable ,
> generally we should block everyone which is what I would prefer but that
> means anyone moving to authorizer must go to all of his existing topics
> and add acl to allow all. If we are fine with imposing this requirement I
> can start returning deny when no acls are found.
> } else {
> //So the user has set some acls explicitly, this means they have
> knowingly enabled authorizer. Let’t first check if they have set an Acl to
> deny this user/host/operation combination.
> if some acl denies this request for this principal/host/operation
> combination , return deny
>
> //this principal/host/operation does not have any explicit deny
> acl,
> check if there is some explicit acl that allows the operation
> if at least one acl allows this request for this
> principal/host/operation
> combination , return allow
>
> // no acl was found for this principal/host/operation combination
> to
> allow this operation, so we will deny the request
> return deny
> }
>
>
> Thanks
> Parth
>
>
> On 4/20/15, 2:21 PM, "Jun Rao" <j...@confluent.io> wrote:
>
> >Hmm, I thought the semantics is that if you only have rule "deny user2",
> >it
> >means that everyone except user2 has access?
> >
> >Thanks,
> >
> >Jun
> >
> >On Mon, Apr 20, 2015 at 3:25 PM, Parth Brahmbhatt <
> >pbrahmbh...@hortonworks.com> wrote:
> >
> >> user3 does not have access and removing the deny rule does not grant him
> >> or user2 access. user2 even without the deny rule will not have access.
> >>
> >> Thanks
> >> Parth
> >>
> >> On 4/20/15, 12:03 PM, "Jun Rao" <j...@confluent.io> wrote:
> >>
> >> >Just a followup question. Suppose there are two rules. Rule1 allows
> >>user1
> >> >and rule2 denies user2. Does user3 have access? If not, does removing
> >> >rule1
> >> >enable user3 access?
> >> >
> >> >Thanks,
> >> >
> >> >Jun
> >> >
> >> >On Mon, Apr 20, 2015 at 1:34 PM, Parth Brahmbhatt <
> >> >pbrahmbh...@hortonworks.com> wrote:
> >> >
> >> >>
> >> >> Hi Joel,
> >> >>
> >> >> Thanks for the review and I plan to update the KIP today with all the
> >> >> updated info. My comments in line below.
> >> >>
> >> >> Thanks
> >> >> Parth
> >> >>
> >> >>
> >> >> On 4/20/15, 10:07 AM, "Joel Koshy" <jjkosh...@gmail.com<mailto:
> >> >> jjkosh...@gmail.com>> wrote:
> >> >>
> >> >> Hi Parth,
> >> >>
> >> >> Nice work on this KIP. I did another read through and had a few more
> >> >> comments (with edits after I went through the thread). Many of these
> >> >> comments were brought up by others as well, so it appears that the
> >>KIP
> >> >> would benefit from an update at this point to incorporate comments
> >> >> from the thread and last hangout.
> >> >>
> >> >> - The operation enum is mostly self-explanatory, but it would help
> >> >> (for the sake of clarity and completeness if nothing else) to
> >> >> document exactly what each of the enums are. E.g., I think this
> >>came
> >> >> up in our hangout - SEND_CONTROL_MESSAGE is unclear and I don't
> >> >> remember what was said about it. <Edit>: After going through the
> >> >> thread it seems the conclusion was to categorize operations. E.g.,
> >> >> WRITE could apply to multiple requests. Again, this is unclear, so
> >> >> if it would be great if you could update the KIP to clarify what
> >>you
> >> >> intend.
> >> >>
> >> >> Will add to document. SEND_CONTROL_MESSAGE Probably a very bad name
> >>but
> >> >> these are intra borker API calls like controller notifying other
> >> >>brokers to
> >> >> update metadata or heartbeats. Any better naming suggestions?
> >> >>
> >> >> - When you update the KIP to categorize the requests it would also
> >> >> help to have a column for what the resource is for each.
> >> >>
> >> >> Will add to the KIP.
> >> >>
> >> >> - FWIW I prefer a 1-1 mapping between requests and operations. I
> >>think
> >> >> categorizing requests into these can be confusing because:
> >> >> - The resource being protected for different requests will be
> >> >> different. We are mostly thinking about topics (read/write) but
> >> >> there are requests for which topic is not the right resource.
> >> >> E.g., for topic creation, the resource as you suggested would be
> >> >> something global/common such as “cluster”. For
> >> >> OffsetCommit/FetchRequest, the resource may be the consumer
> >>group,
> >> >> or maybe a tuple of <consumer group, topic>. So this can be
> >> >> confusing - i.e., different resources and request types in the
> >> >> same category. It may be simpler and clearer to just have a 1-1
> >> >> mapping between the operation enum and requests.
> >> >>
> >> >> I only see 2 resource categories right now cluster and topic. I
> >>don’t
> >> >> really care one way or another so we can probably make a quick
> >>decision
> >> >>in
> >> >> tomorrow’s meeting to either to 1-1 mapping or have categorization?
> >> >>
> >> >> - Some requests that are intuitively READ have WRITE side-effects.
> >> >> E.g., (currently) TopicMetadataRequest with auto-create, although
> >> >> that will eventually go away. ConsumerMetadataRequest still
> >> >> auto-creates the offsets topic. Likewise, ADMIN-type requests may
> >> >> be interpreted as having side-effects (depending on who you ask).
> >> >>
> >> >> Yes and what I am doing right now is checking authorization for all
> >> >> possible actions i.e. for auto-create it checks if the config has it
> >> >> enabled and if yes, check for read + create authorization. Its not
> >>very
> >> >> meaningful right now as there is no CREATE authorization but I think
> >> >>this
> >> >> is implementation detail, we need to ensure we call authorize with
> >>all
> >> >> possible operations from KafkaAPI.
> >> >> - <quote>When an ACL is missing - fail open</quote>. What does
> >>missing
> >> >> mean? i.e., no explicit ACL for a principal? I'm confused by this
> >> >> especially in relation to the precedence of DENY over ALLOW. So per
> >> >> the description:
> >> >> - If no ACLs exist for topic A then ALLOW all operations on it by
> >> >> anyone.
> >> >> - If I now add an ACL for a certain principal P to ALLOW (say)
> >>WRITE
> >> >> to the topic then either:
> >> >> - This has the effect of DENYing WRITE to all other principals
> >> >> - Or, this ACL serves no purpose
> >> >> - If the effect is to DENY WRITE to all other principals, what
> >>about
> >> >> READ. Do all principals (including P) have READ permissions to
> >> >> topic A?
> >> >> - In other words, it seems for a specific ACL to be meaningful then
> >> >> fail close is necessary for an absent ACL.
> >> >> - <edit>After through the thread: it appears that the DENY override
> >> >> only applies to the given principal. i.e., in the above case it
> >> >> appears that the other principals will in fact be granted access.
> >> >> Then this makes the ACL that was added pointless right?
> >> >>
> >> >> The rule I was going with is
> >> >> - If there is no ACL I.e. This might be a topic that was created in
> >>non
> >> >> secure mode or was created before we supported ACLs. We assume you do
> >> >>not
> >> >> want authorization and let all requests go through.
> >> >> - once you add any ACL, we assume you want authorization on the topic
> >> >>and
> >> >> all the general authorization rules now start to apply, I.e we fail
> >> >>close
> >> >> if we don’t find an ACL that allows access or if we find an ACL that
> >> >>denies
> >> >> access. It does not matter if you added a READACL or WRITEACL or
> >> >>ALLOWACL
> >> >> or DENY ACL. If you add any ACL, now every user gets checked against
> >> >>that
> >> >> and if it does not satisfy the ACL, request fails. I.e. If you add an
> >> >>ACL
> >> >> “Allow write to topic-1 form user1 from all hosts” , user-1 has write
> >> >> access from all hosts and no other user has any access(except for
> >> >> superusers who have all the access).
> >> >> - Deny ACLS are suppose to be used to restrict access authorized by
> >>some
> >> >> allow ACL, they are not suppose to be required. Implicitly anyone who
> >> >>does
> >> >> not have an allow acl, gets denied. The Deny ACLs are only added to
> >>give
> >> >> more control to administrators who wants more granular control with
> >> >>lesser
> >> >> config. The scenario described in mailing list was “Allow user X
> >>access
> >> >> from all hosts but Host1,Host2”. in absence of DENY operator you will
> >> >>have
> >> >> to exhaustively list all possible hosts in your ACL which is what we
> >>are
> >> >> trying to avoid.
> >> >>
> >> >> - On ZK ACLs: I think ZK will be closed to everyone except Kafka
> >> >> brokers. This is a dependency on KIP-4 though. i.e., eventually all
> >> >> clients should talk to brokers only via RPC.
> >> >>
> >> >> Yes.
> >> >>
> >> >> - Topic owner: list vs single entry - both have issues off the bat
> >> >> (although list is more intuitive at least to me), but perhaps you
> >> >> could write up some example workflows to clarify the current
> >> >> proposal. I was thinking that anyone in the owner list should be
> >> >> considered a super-user of the topic and can grant/revoke
> >> >> permissions. They should also be allowed to add other principals as
> >> >> owners. Even with this it is unclear who should be allowed to
> >>remove
> >> >> owners.
> >> >>
> >> >> As you pointed out in the last KIP meeting owners/creators have use
> >>out
> >> >> side of security context (plain simple auditing). I don’t think the
> >> >> authorizer work depends on this, it was my bad to even mention it in
> >> >>first
> >> >> place. I think we can have this discussion outside of
> >> >>authorizer/security
> >> >> context and once we have a way to get topic owners the default
> >> >>Authorizer
> >> >> can start using it. It makes sense to treat all owners as super users
> >> >>and I
> >> >> think it is safe to assume superusers can also modify ownership but I
> >> >>think
> >> >> this should not be treated as blocking work for authorization.
> >> >>
> >> >> - What is the effect of deleting a topic - should all associated ACLs
> >> >> be deleted as well?
> >> >> They should be and with acls being stored as part of TopicConfig this
> >> >>was
> >> >> taken care of automatically. With the new ACL management API the
> >>users
> >> >>will
> >> >> have to call remove ACLs explicitly to perform the cleanup. If
> >>everyone
> >> >> thinks this should be automated , with the new APIs we will need a
> >> >>hook(or
> >> >> poll) to be notified when a topic is deleted to perform cleanup.
> >> >> - TopicConfigCache to store topic-ACLs. As mentioned above, not all
> >> >> requests will be tied to topics. We may want to have an entirely
> >> >> separate ZK directory for ACLs. We have a similar issue with
> >>quotas.
> >> >> This ties in with dynamic config management. We can certainly
> >> >> leverage the dynamic config management part of topic configs but I
> >> >> think we need to have a story for non-topic resources.
> >> >>
> >> >> In the first proposal I was going with a topic-Acl and cluster-Acl
> >>where
> >> >> cluster-Acls were json acl local files on all brokers. With the new
> >>ACL
> >> >> management APIs we are planning to have /kafka-acl node under which
> >>all
> >> >> acls will be stored in /kakfa-acls/resource-name -> {acl json data}.
> >> >> Cluster acls will just have resource name kafka-cluster.
> >> >>
> >> >> Thanks,
> >> >>
> >> >> Joel
> >> >>
> >> >> On Thu, Apr 16, 2015 at 12:15:37AM +0000, Parth Brahmbhatt wrote:
> >> >> Kafka currently stores logConfig overrides specified during topic
> >> >>creation
> >> >> in zookeeper, its just an instance of java.util.Properties converted
> >>to
> >> >> json. I am proposing in addition to that we store acls and owner as
> >>well
> >> >> as part of same Properties map.
> >> >> There is some infrastructure around reading this config, converting
> >>it
> >> >> back to Properties map and most importantly propagating any changes
> >> >> efficiently which we will be able to leverage. As this
> >>infrastructure is
> >> >> common to the cluster the reading (not interpreting) of config
> >>happens
> >> >> outside of any authorization code.
> >> >> If the TopicConfigCache just kept the json representation and left
> >>it to
> >> >> authorizer to parse it, the authorizer will have to either parse the
> >> >>json
> >> >> for each request(not acceptable) or it will have to keep one more
> >>layer
> >> >>of
> >> >> parsed ACL instance cache. Assuming authorizer will keep an
> >>additional
> >> >> caching layer we will now have to implement some way to invalidate
> >>the
> >> >> cache which means the TopicConfigCache will have to be an observable
> >> >>which
> >> >> the Authorizer observes and invalidates its cache entries when
> >> >> topicConfigCache gets updated. Seemed like unnecessary complexity
> >>with
> >> >>not
> >> >> lot to gain so I went with TopicConfigCache interpreting the json and
> >> >> caching a higher level modeled object.
> >> >> In summary, the interpretation is done for both optimization and
> >> >> simplicity. If you think it is important to allow custom ACL format
> >> >> support we can add one more pluggable config(acl.parser) and
> >> >> interface(AclParser) or it could just be another method in
> >>Authorizer.
> >> >> One thing to note the current ACL json is versioned so it is easy to
> >> >>make
> >> >> changes to it however it won’t be possible to support custom ACL
> >>formats
> >> >> with the current design.
> >> >> Thanks
> >> >> Parth
> >> >> On 4/15/15, 4:29 PM, "Michael Herstine"
> >><mherst...@linkedin.com.INVALID
> >> >> <mailto:mherst...@linkedin.com.INVALID>>
> >> >> wrote:
> >> >> >Hi Parth,
> >> >> >
> >> >> >I’m a little confused: why would Kafka need to interpret the JSON?
> >> >>IIRC
> >> >> >KIP-11 even says that the TopicConfigData will just store the JSON.
> >>I’m
> >> >> >not really making a design recommendation here, just trying to
> >> >>understand
> >> >> >what you’re proposing.
> >> >> >
> >> >> >On 4/15/15, 11:20 AM, "Parth Brahmbhatt"
> >><pbrahmbh...@hortonworks.com
> >> >> <mailto:pbrahmbh...@hortonworks.com>>
> >> >> >wrote:
> >> >> >
> >> >> >>Hi Michael,
> >> >> >>
> >> >> >>There is code in kafka codebase that reads and interprets the topic
> >> >> >>config JSON which has acls, owner and logconfig properties. There
> >>are
> >> >>3
> >> >> >>use cases that we are supporting with current proposal:
> >> >> >>
> >> >> >> * You use out of box simpleAcl authorizer which is tied to the
> >>acl
> >> >> >>stored in topic config and the format is locked down.
> >> >> >> * You have a custom authorizer and a custom ACL store.
> >> >>Ranger/Argus
> >> >> >>falls under this as they have their own acl store and ui that users
> >> >>use
> >> >> >>to configure acls on the cluster and cluster resources like topic.
> >> >>It is
> >> >> >>upto the custom authorizer to leverage the kafka acl configs or
> >> >> >>completely ignore them as they have set a user expectation that
> >>only
> >> >>acls
> >> >> >>configured via their ui/system will be effective.
> >> >> >> * You have a custom authorizer but no custom Acl store. You are
> >> >> >>completely tied to Acl structure that we have provided in out of
> >>box
> >> >> >>implementation.
> >> >> >>
> >> >> >>Thanks
> >> >> >>Parth
> >> >> >>
> >> >> >>On 4/15/15, 10:31 AM, "Michael Herstine"
> >> >>
> >>>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >> >>wrote:
> >> >> >>
> >> >> >>Hi Parth,
> >> >> >>
> >> >> >>One question that occurred to me at the end of today’s hangout: how
> >> >>tied
> >> >> >>are we to a particular ACL representation under your proposal? I
> >>know
> >> >> >>that
> >> >> >>TopicConfigCache will just contain JSON— if a particular site
> >>decides
> >> >> >>they
> >> >> >>want to represent their ACLs differently, and swap out the
> >>authorizer
> >> >> >>implementation, will that work? I guess what I’m asking is whether
> >> >> >>there’s any code in the Kafka codebase that will interpret that
> >>JSON,
> >> >>or
> >> >> >>does that logic live exclusively in the authorizer?
> >> >> >>
> >> >> >>On 4/14/15, 10:56 PM, "Don Bosco Durai"
> >> >>
> >>>><bo...@apache.org<mailto:bo...@apache.org><mailto:bo...@apache.org>>
> >> >> wrote:
> >> >> >>
> >> >> >>I also feel, having just IP would be more appropriate. Host lookup
> >> >>will
> >> >> >>unnecessary slow things down and would be insecure as you pointed
> >>out.
> >> >> >>
> >> >> >>With IP, it will be also able to setup policies (in future if
> >>needed)
> >> >> >>with
> >> >> >>ranges or netmasks and it would be more scalable.
> >> >> >>
> >> >> >>Bosco
> >> >> >>
> >> >> >>
> >> >> >>On 4/14/15, 1:40 PM, "Michael Herstine"
> >> >>
> >>>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >> >>wrote:
> >> >> >>
> >> >> >>Hi Parth,
> >> >> >>
> >> >> >>Sorry to chime in so late, but I’ve got a minor question on the
> >>KIP.
> >> >> >>
> >> >> >>Several methods take a parameter named “host” of type String. Is
> >>that
> >> >> >>intended to be a hostname, or an IP address? If the former, I’m
> >> >>curious
> >> >> >>as
> >> >> >>to how that’s found (in my experience, when accepting an incoming
> >> >>socket
> >> >> >>connection, you only know the IP address, and there isn’t a way to
> >>map
> >> >> >>that to a hostname without a round trip to a DNS server, which is
> >> >> >>insecure
> >> >> >>anyway).
> >> >> >>
> >> >> >>
> >> >> >>On 3/25/15, 1:07 PM, "Parth Brahmbhatt"
> >> >>
> >> >>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
> >> ><mailto
> >> >>>>:
> >> >> pbrahmbh...@hortonworks.com>>
> >> >> >>wrote:
> >> >> >>
> >> >> >>Hi all,
> >> >> >>
> >> >> >>I have modified the KIP to reflect the recent change request from
> >>the
> >> >> >>reviewers. I have been working on the code and I have the server
> >>side
> >> >> >>code
> >> >> >>for authorization ready. I am now modifying the command line
> >> >>utilities.
> >> >> >>I
> >> >> >>would really appreciate if some of the committers can spend
> >>sometime
> >> >>to
> >> >> >>review the KIP so we can make progress on this.
> >> >> >>
> >> >> >>Thanks
> >> >> >>Parth
> >> >> >>
> >> >> >>On 3/18/15, 2:20 PM, "Michael Herstine"
> >> >>
> >>>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID
> >> >> ><mailto:mherst...@linkedin.com.INVALID>>
> >> >> >>wrote:
> >> >> >>
> >> >> >>Hi Parth,
> >> >> >>
> >> >> >>Thanks! A few questions:
> >> >> >>
> >> >> >>1. Do you want to permit rules in your ACLs that DENY access as
> >>well
> >> >>as
> >> >> >>ALLOW? This can be handy setting up rules that have exceptions.
> >>E.g.
> >> >> >>“Allow principal P to READ resource R from all hosts” with “Deny
> >> >> >>principal
> >> >> >>P READ access to resource R from host H1” in combination would
> >>allow P
> >> >> >>to
> >> >> >>READ R from all hosts *except* H1.
> >> >> >>
> >> >> >>2. When a topic is newly created, will there be an ACL created for
> >>it?
> >> >> >>If
> >> >> >>not, would that not deny subsequent access to it?
> >> >> >>
> >> >> >>(nit) Maybe use Principal instead of String to represent
> >>principals?
> >> >> >>
> >> >> >>
> >> >> >>On 3/9/15, 11:48 AM, "Don Bosco Durai"
> >> >>
> >>>><bo...@apache.org<mailto:bo...@apache.org><mailto:bo...@apache.org>>
> >> >> wrote:
> >> >> >>
> >> >> >>Parth
> >> >> >>
> >> >> >>Overall it is looking good. Couple of questionsŠ
> >> >> >>
> >> >> >>- Can you give an example how the policies will look like in the
> >> >> >>default
> >> >> >>implementation?
> >> >> >>- In the operations, can we support ³CONNECT² also? This can be
> >>used
> >> >> >>during Session connection
> >> >> >>- Regarding access control for ³Topic Creation², since we can¹t do
> >>it
> >> >> >>on
> >> >> >>the server side, can we de-scope it for? And plan it as a future
> >> >> >>feature
> >> >> >>request?
> >> >> >>
> >> >> >>Thanks
> >> >> >>
> >> >> >>Bosco
> >> >> >>
> >> >> >>
> >> >> >>On 3/6/15, 8:10 AM, "Harsha"
> >><ka...@harsha.io<mailto:ka...@harsha.io
> >> >> ><mailto:ka...@harsha.io>>
> >> >> >>wrote:
> >> >> >>
> >> >> >>Hi Parth,
> >> >> >> Thanks for putting this together. Overall it looks good
> >> >> >>to
> >> >> >> me. Although AdminUtils is a concern KIP-4 can
> >>probably
> >> >> >>fix
> >> >> >> that part.
> >> >> >>Thanks,
> >> >> >>Harsha
> >> >> >>
> >> >> >>On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote:
> >> >> >>Forgot to add links to wiki and jira.
> >> >> >>Link to wiki:
> >> >>
> >>>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza
> >> >> >>t
> >> >> >>i
> >> >> >>o
> >> >> >>n
> >> >> >>+
> >> >> >>Interface
> >> >> >>Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688
> >> >> >>Thanks
> >> >> >>Parth
> >> >> >>From: Parth Brahmbhatt
> >> >>
> >> >>>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com
> >> ><mailto
> >> >>>>:
> >> >> pbrahmbh...@hortonworks.com><mailto:p
> >> >> >>b
> >> >> >>rahmbh...@hortonworks.com<mailto:rahmbh...@hortonworks.com>>>
> >> >> >>Date: Thursday, March 5, 2015 at 10:33 AM
> >> >> >>To:
> >> >> >>"dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:
> >> >> dev@kafka.apache.org><mailto:dev@kafka.apach
> >> >> >>e
> >> >> >>.org>"
> >> >> >><dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:
> >> >> dev@kafka.apache.org><mailto:dev@kafka.apach
> >> >> >>e
> >> >> >>.org>>
> >> >> >>Subject: [DISCUSS] KIP-11- Authorization design for kafka security
> >> >> >>Hi,
> >> >> >>KIP-11 is open for discussion , I have updated the wiki with the
> >> >> >>design
> >> >> >>and open questions.
> >> >> >>Thanks
> >> >> >>Parth
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >
> >> >>
> >> >>
> >> >>
> >>
> >>
>
>
