Thanks, SASL setup for ZK does the trick. Now, I am continuing the security setup with schema registry. I am configuring it to use same PLAINTEXT method for now.
listeners=http://0.0.0.0:8081 kafkastore.connection.url=localhost:2181 kafkastore.topic=_schemas debug=true # Security specific params kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092 #kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT://or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092 kafkastore.init.timeout.ms=10000 kafkastore.security.protocol=SASL_PLAINTEXT zookeeper.set.acl=true kafkastore.sasl.mechanism=PLAIN kafkastore.group.id=schema-registry-consumer-group I have followed instructions (modified for my use case and latest version of confluent) from here: https://www.confluent.io/blog/securing-confluent-schema-registry-apache-kafka/ When I try to start schema registry, it fails with following error: [2017-09-06 06:05:23,790] ERROR Server died unexpectedly: (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52) java.lang.IllegalArgumentException: Unable to subscribe to the Kafka topic _schemas backing this data store. Topic may not exist. at io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130) However, I have enabled schema registry user to produce/consume messages for _schemas topic and authorizer debug messages show the same: [2017-09-06 06:05:21,784] DEBUG operation = Read on resource = Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s chemaregistry has Allow permission for operations: All from hosts: * (kafka.authorizer.logger) [2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger) [2017-09-06 06:05:22,788] DEBUG operation = Read on resource = Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:schemaregistry has Allow permission for operations: All from hosts: * (kafka.authorizer.logger) [2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger) Any ideas of what might be happening here? Here're contents of my JAAS file that I use in the startup script: [root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS} -Djava.security.auth.login.config=/etc/schema-registry/schema_registry_client_jaas.properties io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain "$@" [root@or1010050208015 kafka]# cat /etc/schema-registry/schema_registry_client_jaas.properties /* Kafka Client */ KafkaClient { org.apache.kafka.common.security.plain.PlainLoginModule required username="schemaregistry" password="schemaRegistry"; }; /* Zookeeper client */ Client { org.apache.zookeeper.server.auth.DigestLoginModule required username="zk_admin" password="zk_admin_secret"; }; On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <spa...@pdxinc.com> wrote: > Yes, as per my understanding for now the only way to secure ZK is SASL. > https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK > could also be secured using SSL. > > Also remember not use any OS user, anyone user on n\w who can connect to > ZK host:port will be able to modify the ACLs. > > - Shri > > -----Original Message----- > From: Manoj Murumkar [mailto:manoj.murum...@gmail.com] > Sent: Thursday, August 31, 2017 11:03 AM > To: users@kafka.apache.org > Subject: [EXTERNAL] Re: Using ACLs without Kerberos > > ***** Notice: This email was received from an external source ***** > > >> Current kafka-acls.sh script directly contacts zookeeper to create ACLs. > Any OS user who got access to zookeeper can create ACLs for any Kafka > principal. > > Thanks for that point. Appreciate it. > > On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <manikumar.re...@gmail.com> > wrote: > > > There is no correlation between OS user and Kafka Principal/Username. > > Here user name refers to the principal associated with the kafka > > communication channel (Kerberos Principal, SASL/Plain username, Scram > > username, SSL > > certificate) > > > > Current kafka-acls.sh script directly contacts zookeeper to create ACLs. > > Any OS user who got access to zookeeper can create ACLs for any Kafka > > principal. > > This can be restricted by enabling Zk Kerberos authentication. > > http://kafka.apache.org/documentation/#zk_authz > > > > From Kafka 0.11.0.0, We can create ACLs using Java Admin API. > > > > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar > > <manoj.murum...@gmail.com> > > wrote: > > > > > Right, I am. Just to be clear, I am using kafka-acl script to > > define/remove > > > ACLs as a non-super user and it just works fine. I had expected it > > > to > > work > > > only for super users and not for regular users ('nex37045' is a > > > normal user). > > > > > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer > > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties > > > zookeeper.connect=localhost:2181 --list Current ACLs for resource > > > `Topic:test`: > > > User:nex37045 has Deny permission for operations: Write from > hosts: > > * > > > User:skumarmu has Allow permission for operations: Write from > hosts: > > * > > > > > > [nex37045@or1010051029033 ~]$ > > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer > > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties > > > zookeeper.connect=localhost:2181 --remove --allow-principal > > User:skumarmu > > > --operation Write --topic test > > > Are you sure you want to remove ACLs: > > > User:skumarmu has Allow permission for operations: Write from > > hosts: * > > > from resource `Topic:test`? (y/n) > > > y > > > Current ACLs for resource `Topic:test`: > > > User:nex37045 has Deny permission for operations: Write from > hosts: > > * > > > > > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer > > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties > > > zookeeper.connect=localhost:2181 --list Current ACLs for resource > > > `Topic:test`: > > > User:nex37045 has Deny permission for operations: Write from > hosts: > > * > > > > > > > > > > > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar > > > <manikumar.re...@gmail.com> > > > wrote: > > > > > > > Looks like you are already using SASL/PLAIN mechanism. Kafka > > > > supports > > > SASL > > > > authentication framework. > > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. > > > > you > > can > > > > enable SSL encryption also > > > > > > > > http://kafka.apache.org/documentation.html#security > > > > > > > > > > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar < > > > manoj.murum...@gmail.com> > > > > wrote: > > > > > > > > > Thanks Manikumar. I am testing the setup documented here: > > > > > https://developer.ibm.com/opentech/2017/05/31/kafka- > > acls-in-practice/ > > > > > (SASL_PLAINTEXT). > > > > > > > > > > I haven't setup any authentication for the tests. Thinking about > > > > > it, authentication is a must have for authorization (so, kafka > > > > > knows > > who's > > > > > making resource request), right? > > > > > > > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar < > > manikumar.re...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Kafka default authorizer is used with secure authenticated > > > > > > channels (SSL,SASL,SCRAM). > > > > > > For plain text (non-secure) channels, the principal will be > > > > > > always ANONYMOUS. Here you can authorize by ip-address. > > > > > > > > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN > > > > > > or > > > SCRAM > > > > > > mechanisms with/without SSL. > > > > > > Pl, check Kafka docs for security considerations. > > > > > > > > > > > > >> I was testing if a non-admin principal (OS user) can modify > > > > > (add/remove) > > > > > > ACLs and > > > > > > it seems like it's possible. > > > > > > How are you trying? Can you give more details about your setup? > > What > > > > > > authentication mechanism, etc.? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar < > > > > > manoj.murum...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > We are evaluating how to put authorization in place for > > > > > > > Kafka > > > (around > > > > > > > topics, mostly). Is it a good idea to do this without > > > > > > > Kerberos? I > > > was > > > > > > > testing if a non-admin principal (OS user) can modify > > (add/remove) > > > > ACLs > > > > > > and > > > > > > > it seems like it's possible. If this is right behavior, it's > > > insecure > > > > > and > > > > > > > unusable. What do you guys think? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Manoj > > > > > > > > > > > > > > > > > > > > > > > > > > > > This e-mail and its contents (to include attachments) are the property of > National Health Systems, Inc., its subsidiaries and affiliates, including > but not limited to Rx.com Community Healthcare Network, Inc. and its > subsidiaries, and may contain confidential and proprietary or privileged > information. If you are not the intended recipient of this e-mail, you are > hereby notified that any unauthorized disclosure, copying, or distribution > of this e-mail or of its attachments, or the taking of any unauthorized > action based on information contained herein is strictly prohibited. > Unauthorized use of information contained herein may subject you to civil > and criminal prosecution and penalties. If you are not the intended > recipient, please immediately notify the sender by telephone at > 800-433-5719 or return e-mail and permanently delete the original e-mail. >
