Hi Parth, OK, I understand your approach much better, now. I don’t really have a strong opinion; I guess I just shared Gwen’s concern (in the hangout) that, with this approach, if a site implements their own authorizer using their own ACL implementation, it will work, but they’ll have this vestigial infrastructure for creating & maintaining ACLs that could cause confusion.
One final thought: whatever the abstraction you’re planning for to represent an ACL in the TopicConfig, could you make that pluggable? On 4/15/15, 5:15 PM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com> wrote: >Kafka currently stores logConfig overrides specified during topic creation >in zookeeper, its just an instance of java.util.Properties converted to >json. I am proposing in addition to that we store acls and owner as well >as part of same Properties map. >There is some infrastructure around reading this config, converting it >back to Properties map and most importantly propagating any changes >efficiently which we will be able to leverage. As this infrastructure is >common to the cluster the reading (not interpreting) of config happens >outside of any authorization code. > >If the TopicConfigCache just kept the json representation and left it to >authorizer to parse it, the authorizer will have to either parse the json >for each request(not acceptable) or it will have to keep one more layer of >parsed ACL instance cache. Assuming authorizer will keep an additional >caching layer we will now have to implement some way to invalidate the >cache which means the TopicConfigCache will have to be an observable which >the Authorizer observes and invalidates its cache entries when >topicConfigCache gets updated. Seemed like unnecessary complexity with not >lot to gain so I went with TopicConfigCache interpreting the json and >caching a higher level modeled object. > >In summary, the interpretation is done for both optimization and >simplicity. If you think it is important to allow custom ACL format >support we can add one more pluggable config(acl.parser) and >interface(AclParser) or it could just be another method in Authorizer. >One thing to note the current ACL json is versioned so it is easy to make >changes to it however it won’t be possible to support custom ACL formats >with the current design. > >Thanks >Parth > >On 4/15/15, 4:29 PM, "Michael Herstine" <mherst...@linkedin.com.INVALID> >wrote: > >>Hi Parth, >> >>I’m a little confused: why would Kafka need to interpret the JSON? IIRC >>KIP-11 even says that the TopicConfigData will just store the JSON. I’m >>not really making a design recommendation here, just trying to understand >>what you’re proposing. >> >>On 4/15/15, 11:20 AM, "Parth Brahmbhatt" <pbrahmbh...@hortonworks.com> >>wrote: >> >>>Hi Michael, >>> >>>There is code in kafka codebase that reads and interprets the topic >>>config JSON which has acls, owner and logconfig properties. There are 3 >>>use cases that we are supporting with current proposal: >>> >>> * You use out of box simpleAcl authorizer which is tied to the acl >>>stored in topic config and the format is locked down. >>> * You have a custom authorizer and a custom ACL store. Ranger/Argus >>>falls under this as they have their own acl store and ui that users use >>>to configure acls on the cluster and cluster resources like topic. It >>>is >>>upto the custom authorizer to leverage the kafka acl configs or >>>completely ignore them as they have set a user expectation that only >>>acls >>>configured via their ui/system will be effective. >>> * You have a custom authorizer but no custom Acl store. You are >>>completely tied to Acl structure that we have provided in out of box >>>implementation. >>> >>>Thanks >>>Parth >>> >>>On 4/15/15, 10:31 AM, "Michael Herstine" >>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID>> >>>wrote: >>> >>>Hi Parth, >>> >>>One question that occurred to me at the end of today’s hangout: how tied >>>are we to a particular ACL representation under your proposal? I know >>>that >>>TopicConfigCache will just contain JSON— if a particular site decides >>>they >>>want to represent their ACLs differently, and swap out the authorizer >>>implementation, will that work? I guess what I’m asking is whether >>>there’s any code in the Kafka codebase that will interpret that JSON, or >>>does that logic live exclusively in the authorizer? >>> >>>On 4/14/15, 10:56 PM, "Don Bosco Durai" >>><bo...@apache.org<mailto:bo...@apache.org>> wrote: >>> >>>I also feel, having just IP would be more appropriate. Host lookup will >>>unnecessary slow things down and would be insecure as you pointed out. >>> >>>With IP, it will be also able to setup policies (in future if needed) >>>with >>>ranges or netmasks and it would be more scalable. >>> >>>Bosco >>> >>> >>>On 4/14/15, 1:40 PM, "Michael Herstine" >>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID>> >>>wrote: >>> >>>Hi Parth, >>> >>>Sorry to chime in so late, but I’ve got a minor question on the KIP. >>> >>>Several methods take a parameter named “host” of type String. Is that >>>intended to be a hostname, or an IP address? If the former, I’m curious >>>as >>>to how that’s found (in my experience, when accepting an incoming socket >>>connection, you only know the IP address, and there isn’t a way to map >>>that to a hostname without a round trip to a DNS server, which is >>>insecure >>>anyway). >>> >>> >>>On 3/25/15, 1:07 PM, "Parth Brahmbhatt" >>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com>> >>>wrote: >>> >>>Hi all, >>> >>>I have modified the KIP to reflect the recent change request from the >>>reviewers. I have been working on the code and I have the server side >>>code >>>for authorization ready. I am now modifying the command line utilities. >>>I >>>would really appreciate if some of the committers can spend sometime to >>>review the KIP so we can make progress on this. >>> >>>Thanks >>>Parth >>> >>>On 3/18/15, 2:20 PM, "Michael Herstine" >>><mherst...@linkedin.com.INVALID<mailto:mherst...@linkedin.com.INVALID>> >>>wrote: >>> >>>Hi Parth, >>> >>>Thanks! A few questions: >>> >>>1. Do you want to permit rules in your ACLs that DENY access as well as >>>ALLOW? This can be handy setting up rules that have exceptions. E.g. >>>“Allow principal P to READ resource R from all hosts” with “Deny >>>principal >>>P READ access to resource R from host H1” in combination would allow P >>>to >>>READ R from all hosts *except* H1. >>> >>>2. When a topic is newly created, will there be an ACL created for it? >>>If >>>not, would that not deny subsequent access to it? >>> >>>(nit) Maybe use Principal instead of String to represent principals? >>> >>> >>>On 3/9/15, 11:48 AM, "Don Bosco Durai" >>><bo...@apache.org<mailto:bo...@apache.org>> wrote: >>> >>>Parth >>> >>>Overall it is looking good. Couple of questionsŠ >>> >>>- Can you give an example how the policies will look like in the >>>default >>>implementation? >>>- In the operations, can we support ³CONNECT² also? This can be used >>>during Session connection >>>- Regarding access control for ³Topic Creation², since we can¹t do it >>>on >>>the server side, can we de-scope it for? And plan it as a future >>>feature >>>request? >>> >>>Thanks >>> >>>Bosco >>> >>> >>>On 3/6/15, 8:10 AM, "Harsha" <ka...@harsha.io<mailto:ka...@harsha.io>> >>>wrote: >>> >>>Hi Parth, >>> Thanks for putting this together. Overall it looks good >>>to >>> me. Although AdminUtils is a concern KIP-4 can probably >>>fix >>> that part. >>>Thanks, >>>Harsha >>> >>>On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: >>>Forgot to add links to wiki and jira. >>>Link to wiki: >>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza >>>t >>>i >>>o >>>n >>>+ >>>Interface >>>Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 >>>Thanks >>>Parth >>>From: Parth Brahmbhatt >>><pbrahmbh...@hortonworks.com<mailto:pbrahmbh...@hortonworks.com><mailto: >>>p >>>b >>>rahmbh...@hortonworks.com>> >>>Date: Thursday, March 5, 2015 at 10:33 AM >>>To: >>>"dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:d...@kafka.apac >>>h >>>e >>>.org>" >>><dev@kafka.apache.org<mailto:dev@kafka.apache.org><mailto:d...@kafka.apac >>>h >>>e >>>.org>> >>>Subject: [DISCUSS] KIP-11- Authorization design for kafka security >>>Hi, >>>KIP-11 is open for discussion , I have updated the wiki with the >>>design >>>and open questions. >>>Thanks >>>Parth >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >
