Hi Joris, I have forwarded your mail to *secur...@confluent.io <secur...@confluent.io>* since it is about security in the Confluent Platform rather than in Apache Kafka.
Regards, Rajini On Tue, Sep 17, 2019 at 11:35 AM Joris Peeters <joris.mg.peet...@gmail.com> wrote: > Hello, > > I am trying to come up with a good security approach for a Kafka project > inside our company. I see there's a variety of options available > (ACL/RBAC/certificates/...), and I'm hoping someone can suggest a few > possibilities. > Some points of interest, > > - Kafka will be used both by end-users/researchers (inside the company) and > by some microservices. The users are on a wide variety of platforms > (Windows/Linux, JVM/C++/Python/R/Javascript/MATLAB/...). The microservices > are probably JVM/Python. > - We'd like to make heavy use of both the Schema Registry and REST proxy, > for all users, > - Users & groups are in LDAP, > - Permissioning would need to be granular and at the level of Kafka > (topics, consumer groups, ...), the Schema Registry, the REST Proxy, ... - > based on LDAP group membership. > - Ideally we'd like to use user/password authentication throughout, which > is the most widely supported mechanism (as we use many different > platforms). The user is the LDAP username, but the password is an API key > that users manage separately. For security reasons we cannot use the LDAP > password, as it would be too likely to leak (e.g. in git etc). (We already > use the LDAP user + API key combo for authentication in other places). > > For Kafka, I've played around with an approach similar to > https://github.com/navikt/kafka-plain-saslserver-2-ad, i.e. writing > classes > that extend > org.apache.kafka.common.security.auth.AuthenticateCallbackHandler and > kafka.security.auth.Authorizer for providing our own rules, which works > perfectly fine. In this case, > - the authenticator checks the username against our own database of (user > -> [apiKeys]), > - the authoriser meaningfully overrides only the "authorize(..)" function > (leaving all the *Acl ones as no-ops) and leverages LDAP (in particular > group membership) to decide upon authorization, using an additional > database encoding group permissions. > This is a bit hacky - and more a proof-of-concept than anything else, but > it does seem to cover our needs, as far as Kafka goes. > > However, I believe this is only available for Kafka itself, and a similar > approach wouldn't extend to the REST proxy and the Schema Registry (and > Connectors and the Control Center). Ideally, we want a centralized auth > service. > In that context, I've been looking at Role-Based Access Control. I think it > would suit the majority of our purposes, except for the fact that it seems > to authenticate using LDAP only (using LDAP user/pass for authentication), > which we cannot do. I was wondering, > > - Is it possible to extend the RBAC with a custom authenticator, that maps > (user+passw) -> bool? We could probably still use LDAP directly for > authorisation. If not, though - is a similar class extension possible for > authorisation as well? If I understood correctly, this would need to be > done inside the Metadata Service. > - An alternative could be to run a separate LDAP server, identical to our > company-central one, but where the user passwords are set to their API > keys. That would be quite a faff, though. > > If this is not possible - what is it that makes our use case different from > the norm? Is it that, > - generally people are OK with passing LDAP passwords? > - generally "external" users don't need the schema registry or REST proxy > (i.e. they are only used by internal microservices), so they can be > firewalled off? > - generally all enterprise consumers are on the JVM, making Kerberos auth > more plausible than in our case? > - ... ? > > Note that I do not have a ton of experience with securing Kafka, so the > above may well contain numerous mistaken/out-of-date assumptions. In any > case, thanks for any suggestions. > > Best, > -Joris. >
