This is fixed as well, btw. My ZK config wasn't done properly. On Tue, Sep 5, 2017 at 11:08 PM, Manoj Murumkar <manoj.murum...@gmail.com> wrote:
> Thanks, SASL setup for ZK does the trick. > > Now, I am continuing the security setup with schema registry. I am > configuring it to use same PLAINTEXT method for now. > > listeners=http://0.0.0.0:8081 > kafkastore.connection.url=localhost:2181 > kafkastore.topic=_schemas > debug=true > # Security specific params > kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092 > #kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_ > PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT:// > or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092 > kafkastore.init.timeout.ms=10000 > kafkastore.security.protocol=SASL_PLAINTEXT > zookeeper.set.acl=true > kafkastore.sasl.mechanism=PLAIN > kafkastore.group.id=schema-registry-consumer-group > > I have followed instructions (modified for my use case and latest version > of confluent) from here: https://www.confluent.io/blog/ > securing-confluent-schema-registry-apache-kafka/ > > When I try to start schema registry, it fails with following error: > > [2017-09-06 06:05:23,790] ERROR Server died unexpectedly: > (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52) > java.lang.IllegalArgumentException: Unable to subscribe to the Kafka > topic _schemas backing this data store. Topic may not exist. > at io.confluent.kafka.schemaregistry.storage. > KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130) > > > However, I have enabled schema registry user to produce/consume messages > for _schemas topic and authorizer debug messages show the same: > > [2017-09-06 06:05:21,784] DEBUG operation = Read on resource = > Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s > chemaregistry has Allow permission for operations: All from hosts: * > (kafka.authorizer.logger) > [2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed > Operation = Describe from host = 127.0.0.1 on resource > = Topic:_schemas (kafka.authorizer.logger) > [2017-09-06 06:05:22,788] DEBUG operation = Read on resource = > Topic:_schemas from host = 127.0.0.1 is Allow based on acl = > User:schemaregistry has Allow permission for operations: All from hosts: * > (kafka.authorizer.logger) > [2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed > Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas > (kafka.authorizer.logger) > > Any ideas of what might be happening here? > > Here're contents of my JAAS file that I use in the startup script: > > > [root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start > exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS} > -Djava.security.auth.login.config=/etc/schema-registry/ > schema_registry_client_jaas.properties io.confluent.kafka. > schemaregistry.rest.SchemaRegistryMain "$@" > > > [root@or1010050208015 kafka]# cat /etc/schema-registry/schema_ > registry_client_jaas.properties > /* Kafka Client */ > KafkaClient { > org.apache.kafka.common.security.plain.PlainLoginModule required > username="schemaregistry" > password="schemaRegistry"; > }; > > /* Zookeeper client */ > Client { > org.apache.zookeeper.server.auth.DigestLoginModule required > username="zk_admin" > password="zk_admin_secret"; > }; > > > On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <spa...@pdxinc.com> wrote: > >> Yes, as per my understanding for now the only way to secure ZK is SASL. >> https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK >> could also be secured using SSL. >> >> Also remember not use any OS user, anyone user on n\w who can connect to >> ZK host:port will be able to modify the ACLs. >> >> - Shri >> >> -----Original Message----- >> From: Manoj Murumkar [mailto:manoj.murum...@gmail.com] >> Sent: Thursday, August 31, 2017 11:03 AM >> To: users@kafka.apache.org >> Subject: [EXTERNAL] Re: Using ACLs without Kerberos >> >> ***** Notice: This email was received from an external source ***** >> >> >> Current kafka-acls.sh script directly contacts zookeeper to create >> ACLs. >> Any OS user who got access to zookeeper can create ACLs for any Kafka >> principal. >> >> Thanks for that point. Appreciate it. >> >> On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <manikumar.re...@gmail.com> >> wrote: >> >> > There is no correlation between OS user and Kafka Principal/Username. >> > Here user name refers to the principal associated with the kafka >> > communication channel (Kerberos Principal, SASL/Plain username, Scram >> > username, SSL >> > certificate) >> > >> > Current kafka-acls.sh script directly contacts zookeeper to create ACLs. >> > Any OS user who got access to zookeeper can create ACLs for any Kafka >> > principal. >> > This can be restricted by enabling Zk Kerberos authentication. >> > http://kafka.apache.org/documentation/#zk_authz >> > >> > From Kafka 0.11.0.0, We can create ACLs using Java Admin API. >> > >> > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar >> > <manoj.murum...@gmail.com> >> > wrote: >> > >> > > Right, I am. Just to be clear, I am using kafka-acl script to >> > define/remove >> > > ACLs as a non-super user and it just works fine. I had expected it >> > > to >> > work >> > > only for super users and not for regular users ('nex37045' is a >> > > normal user). >> > > >> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer >> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties >> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource >> > > `Topic:test`: >> > > User:nex37045 has Deny permission for operations: Write from >> hosts: >> > * >> > > User:skumarmu has Allow permission for operations: Write from >> hosts: >> > * >> > > >> > > [nex37045@or1010051029033 ~]$ >> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer >> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties >> > > zookeeper.connect=localhost:2181 --remove --allow-principal >> > User:skumarmu >> > > --operation Write --topic test >> > > Are you sure you want to remove ACLs: >> > > User:skumarmu has Allow permission for operations: Write from >> > hosts: * >> > > from resource `Topic:test`? (y/n) >> > > y >> > > Current ACLs for resource `Topic:test`: >> > > User:nex37045 has Deny permission for operations: Write from >> hosts: >> > * >> > > >> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer >> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties >> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource >> > > `Topic:test`: >> > > User:nex37045 has Deny permission for operations: Write from >> hosts: >> > * >> > > >> > > >> > > >> > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar >> > > <manikumar.re...@gmail.com> >> > > wrote: >> > > >> > > > Looks like you are already using SASL/PLAIN mechanism. Kafka >> > > > supports >> > > SASL >> > > > authentication framework. >> > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. >> > > > you >> > can >> > > > enable SSL encryption also >> > > > >> > > > http://kafka.apache.org/documentation.html#security >> > > > >> > > > >> > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar < >> > > manoj.murum...@gmail.com> >> > > > wrote: >> > > > >> > > > > Thanks Manikumar. I am testing the setup documented here: >> > > > > https://developer.ibm.com/opentech/2017/05/31/kafka- >> > acls-in-practice/ >> > > > > (SASL_PLAINTEXT). >> > > > > >> > > > > I haven't setup any authentication for the tests. Thinking about >> > > > > it, authentication is a must have for authorization (so, kafka >> > > > > knows >> > who's >> > > > > making resource request), right? >> > > > > >> > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar < >> > manikumar.re...@gmail.com >> > > > >> > > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > Kafka default authorizer is used with secure authenticated >> > > > > > channels (SSL,SASL,SCRAM). >> > > > > > For plain text (non-secure) channels, the principal will be >> > > > > > always ANONYMOUS. Here you can authorize by ip-address. >> > > > > > >> > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN >> > > > > > or >> > > SCRAM >> > > > > > mechanisms with/without SSL. >> > > > > > Pl, check Kafka docs for security considerations. >> > > > > > >> > > > > > >> I was testing if a non-admin principal (OS user) can modify >> > > > > (add/remove) >> > > > > > ACLs and >> > > > > > it seems like it's possible. >> > > > > > How are you trying? Can you give more details about your setup? >> > What >> > > > > > authentication mechanism, etc.? >> > > > > > >> > > > > > >> > > > > > Thanks, >> > > > > > >> > > > > > >> > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar < >> > > > > manoj.murum...@gmail.com> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi, >> > > > > > > >> > > > > > > We are evaluating how to put authorization in place for >> > > > > > > Kafka >> > > (around >> > > > > > > topics, mostly). Is it a good idea to do this without >> > > > > > > Kerberos? I >> > > was >> > > > > > > testing if a non-admin principal (OS user) can modify >> > (add/remove) >> > > > ACLs >> > > > > > and >> > > > > > > it seems like it's possible. If this is right behavior, it's >> > > insecure >> > > > > and >> > > > > > > unusable. What do you guys think? >> > > > > > > >> > > > > > > Thanks, >> > > > > > > >> > > > > > > Manoj >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> This e-mail and its contents (to include attachments) are the property of >> National Health Systems, Inc., its subsidiaries and affiliates, including >> but not limited to Rx.com Community Healthcare Network, Inc. and its >> subsidiaries, and may contain confidential and proprietary or privileged >> information. If you are not the intended recipient of this e-mail, you are >> hereby notified that any unauthorized disclosure, copying, or distribution >> of this e-mail or of its attachments, or the taking of any unauthorized >> action based on information contained herein is strictly prohibited. >> Unauthorized use of information contained herein may subject you to civil >> and criminal prosecution and penalties. If you are not the intended >> recipient, please immediately notify the sender by telephone at >> 800-433-5719 or return e-mail and permanently delete the original e-mail. >> > >
