On Thu, Apr 16, 2015 at 6:13 PM, Jun Rao <j...@confluent.io> wrote: > Hi, Gwen, > > What you suggested seems reasonable. I guess we will need the <Principal, > Privilege> pair and the Resource in grant() and revoke()?
I thought that Privilege is a Resource+Action, which is why grant and revoke can take list of principals and list of privileges. But as I said, anything that creates an ACL will work there. > Is the Hive authorization api the following? It's weird that it takes user > in checkPermissions(), but not in authorize(). > > http://hive.apache.org/javadocs/r0.11.0/api/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.html I was thinking about this portion of the API: https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java > I was imagining that a default implementation could be similar to how we > store offsets in Kafka. Basically, store all acls in a special topic with > compact retention. Then, every broker will build an in-memory cache off > that topic. The default authorizer can definitely do it. My suggestion is that this is something the authorizer should worry about, not core Kafka and not the Kafka tools. TopicAdmin and TopicMetadata APIs should just invoke Authorizer.grant(), Authorizer.revoke(), Authorizer.show(), etc. > Another thing that we haven't discussed so far is how to manage ACLs across > different mirrored Kafka clusters. Let's say you use mirror maker to mirror > all topics from cluster A to cluster B. You probably want to have exactly > the same ACL on both A and B. It would be good if the ACL can be set up > just once. If we use the above default implementation, since the ACL topic > is mirrored too, the ACL will be propagated automatically. This is a great suggestion. Currently it is a huge pain-point in Hadoop. Obviously, this will only work if we know that all ACLs are in a Kafka topic, which may not be true for Sentry / Argus. I think its OK for MirrorMaker to support this only for DefaultAuthorizer and force Sentry/Argus to figure out their own replication. But it will also be awesome if we can come up with a way to replicate privileges that relies only on generic APIs and authorizer implementations of them. Gwen > Thanks, > > Jun > > > On Thu, Apr 16, 2015 at 9:44 AM, Gwen Shapira <gshap...@cloudera.com> wrote: > >> Hi Kafka Authorization Fans, >> >> I'm starting a new thread on a specific sub-topic of KIP-11, since >> this is a bit long :) >> >> Currently KIP-11, as I understand it, proposes: >> * Authorizers are pluggable, with Kafka providing DefaultAuthorizer. >> * Kafka tools allow adding / managing ACLs. >> * Those ACLs are stored in ZK and cached in a new TopicCache >> * Authorizers can either use the ACLs defined and stored in Kafka, or >> define and use their own. >> >> I am concerned of two possible issues with this design: >> 1. Separation of concerns - only authorizers should worry about ACLs, >> and therefore the less code for ACLs that exist in Kafka core, the >> better. >> 2. User confusion - It sounded like we can define ACLs in Kafka itself >> but authorizers can also define their own, so "kafka-topics >> --describe" may show an ACL different than the one in use. This can be >> super confusing for admins. >> >> My alternative suggestion: >> * Authorizer API will include: >> grantPrivilege(List<Principals>, List<Privilege>) >> revokePrivilege(List<Principals>, List<Privilege>), >> getPrivilegesByPrincipal(Principal, Resource) >> .... >> (The exact API can be discussed in detail, but you get the idea) >> * Kafka tools will simply invoke these APIs when topics are added / >> modified / described. >> * Each authorizer (including the default one) will be responsible for >> storing, caching and using those ACLs. >> >> This way, we keep almost all ACL code with the Authorizer, where it >> belongs and users get a nice unified interface that reflects what is >> actually getting used in the system. >> This is pretty much how Sqoop and Hive implement their authorization APIs. >> >> What do you think? >> >> Gwen >>
