Hey Raghav, Yes, I would very much love to get your configs, so I can model against it.
Thanks again, Alex From: Raghav <raghavas...@gmail.com> Date: Thursday, May 25, 2017 at 10:54 PM To: Mike Marzo <precisionarchery...@gmail.com> Cc: Darshan Purandare <purandare.dars...@gmail.com>, Rajini Sivaram <rajinisiva...@gmail.com>, Users <users@kafka.apache.org>, Alex Kamalov <alex.kama...@bnymellon.com> Subject: Re: Kafka Authorization and ACLs Broken In SSL cert, there is a field which has a CN (Common Name). So when ACLs are set, they are set for that CN. This is how the ACLs are configured and matched against. I am still pretty new to Kafka in general, but this is how I think it works. I can copy my config if you want. On Thu, May 25, 2017 at 12:51 PM, Mike Marzo <precisionarchery...@gmail.com<mailto:precisionarchery...@gmail.com>> wrote: Stupid question.... If u don't specify a jaas file how does the consumer and producer specify the Id that acl's are configured against.... boy I am getting more and more perplexed by this... mike marzo 908 209-4484<tel:(908)%20209-4484> On May 24, 2017 9:29 PM, "Raghav" <raghavas...@gmail.com<mailto:raghavas...@gmail.com>> wrote: Mike I am not using jaas file. I literally took the config Rajini gave in the previous email and it worked for me. I am using ssl Kafka with ACLs. I am not suing kerberos. Thanks. On Wed, May 24, 2017 at 11:29 AM, Mike Marzo <precisionarchery...@gmail.com<mailto:precisionarchery...@gmail.com>> wrote: I'm also having issues getting acls to work. Out of intereat, are you starting ur brokers with a jaas file, if so do u mind sharing the client and server side jaas entries so I can validate what I'm doing. mike marzo 908 209-4484<tel:908%20209-4484> On May 24, 2017 10:54 AM, "Raghav" <raghavas...@gmail.com<mailto:raghavas...@gmail.com>> wrote: > Hi Rajini > > Thank you very much. It perfectly works. > > I think in my setup I was trying to use a CA (certificate authority) to > sign the certificates from client and server, and then adding it to trust > store and keystore. I think in that process, I may have messed something. I > will try above config with a CA to sign certificates. Hopefully that would > work too. > > Thanks a lot again. > > Raghav > > > > > On Wed, May 24, 2017 at 7:04 AM, Rajini Sivaram > <rajinisiva...@gmail.com<mailto:rajinisiva...@gmail.com>> > wrote: > > > Raghav/Darshan, > > > > Can you try these steps on a clean installation of Kafka? It works for > me, > > so hopefully it will work for you. And then you can adapt to your > scenario. > > > > *Create keystores and truststores:* > > > > keytool -genkey -alias kafka -keystore server.keystore.jks -dname > > "CN=KafkaBroker,O=Pivotal,C=UK" -storepass server-keystore-password > > -keypass server-key-password > > > > keytool -exportcert -file server-cert-file -keystore server.keystore.jks > > -alias kafka -storepass server-keystore-password > > > > keytool -importcert -file server-cert-file -keystore > server.truststore.jks > > -alias kafka -storepass server-truststore-password -noprompt > > > > keytool -importcert -file server-cert-file -keystore > client.truststore.jks > > -alias kafkaclient -storepass client-truststore-password -noprompt > > > > > > keytool -genkey -alias kafkaclient -keystore client.keystore.jks -dname > > "CN=KafkaClient,O=Pivotal,C=UK" -storepass client-keystore-password > > -keypass client-key-password > > > > keytool -exportcert -file client-cert-file -keystore client.keystore.jks > > -alias kafkaclient -storepass client-keystore-password > > > > keytool -importcert -file client-cert-file -keystore > server.truststore.jks > > -alias kafkaclient -storepass server-truststore-password -noprompt > > > > *Configure broker: Add these lines at the end of your server.properties* > > > > listeners=SSL://:9093 > > > > advertised.listeners=SSL://127.0.0.1:9093<http://127.0.0.1:9093> > > > > ssl.keystore.location=/tmp/acl/server.keystore.jks > > > > ssl.keystore.password=server-keystore-password > > > > ssl.key.password=server-key-password > > > > ssl.truststore.location=/tmp/acl/server.truststore.jks > > > > ssl.truststore.password=server-truststore-password > > > > security.inter.broker.protocol=SSL > > > > security.protocol=SSL > > > > ssl.client.auth=required > > > > allow.everyone.if.no.acl.found=false > > > > authorizer.class.name<http://authorizer.class.name>=kafka.se<http://kafka.se>curity.auth.SimpleAclAuthorizer > > > > super.users=User:CN=KafkaBroker,O=Pivotal,C=UK > > > > *Configure producer: producer.properties* > > > > security.protocol=SSL > > > > ssl.truststore.location=/tmp/acl/client.truststore.jks > > > > ssl.truststore.password=client-truststore-password > > > > ssl.keystore.location=/tmp/acl/client.keystore.jks > > > > ssl.keystore.password=client-keystore-password > > > > ssl.key.password=client-key-password > > > > > > *Configure consumer: consumer.properties* > > > > security.protocol=SSL > > > > ssl.truststore.location=/tmp/acl/client.truststore.jks > > > > ssl.truststore.password=client-truststore-password > > > > ssl.keystore.location=/tmp/acl/client.keystore.jks > > > > ssl.keystore.password=client-keystore-password > > > > ssl.key.password=client-key-password > > > > group.id<http://group.id>=testgroup > > > > *Create topic:* > > > > bin/kafka-topics.sh --zookeeper localhost --create --topic testtopic > > --replication-factor 1 --partitions 1 > > > > > > *Configure ACLs:* > > > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: > 2181 > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" --producer > > --topic testtopic > > > > bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost: > 2181 > > --add --allow-principal "User:CN=KafkaClient,O=Pivotal,C=UK" --consumer > > --topic testtopic --group test group > > > > > > *Run console producer and type in some messages:* > > > > bin/kafka-console-producer.sh --producer.config > > /tmp/acl/producer.properties --topic testtopic --broker-list > > 127.0.0.1:9093<http://127.0.0.1:9093> > > > > > > *Run console consumer, you should see messages from above:* > > > > bin/kafka-console-consumer.sh --consumer.config > > /tmp/acl/consumer.properties --topic testtopic --bootstrap-server > > 127.0.0.1:9093<http://127.0.0.1:9093> --from-beginning > > > > > > > > On Tue, May 23, 2017 at 12:57 PM, Raghav > > <raghavas...@gmail.com<mailto:raghavas...@gmail.com>> wrote: > > > >> Darshan, > >> > >> I have not yet successfully gotten the ACLs to work in Kafka. I am still > >> looking for help. I will update this email thread if I do find. In case > >> you > >> get it working, please let me know. > >> > >> Thanks. > >> > >> R > >> > >> On Tue, May 23, 2017 at 8:49 AM, Darshan Purandare < > >> purandare.dars...@gmail.com<mailto:purandare.dars...@gmail.com>> wrote: > >> > >> > Raghav > >> > > >> > I saw few posts of yours around Kafka ACLs and the problems. I have > seen > >> > similar issues where Writer has not been able to write to any topic. I > >> have > >> > seen "leader not available" and sometimes "unknown topic or > partition", > >> and > >> > "topic_authorization_failed" error. > >> > > >> > Let me know if you find a valid config that works. > >> > > >> > Thanks. > >> > > >> > > >> > > >> > On Tue, May 23, 2017 at 8:44 AM, Raghav > >> > <raghavas...@gmail.com<mailto:raghavas...@gmail.com>> > wrote: > >> > > >> >> Hello Kafka Users > >> >> > >> >> I am a new Kafka user and trying to make Kafka SSL work with > >> Authorization > >> >> and ACLs. I followed posts from Kafka and Confluent docs exactly to > the > >> >> point but my producer cannot write to kafka broker. I get > >> >> "LEADER_NOT_FOUND" errors. And even Consumer throws the same errors. > >> >> > >> >> Can someone please share their config which worked with ACLs. > >> >> > >> >> Here is my config. Please help. > >> >> > >> >> server.properties config > >> >> ------------------------------------------------------------ > >> >> ------------------------------------------------ > >> >> broker.id<http://broker.id>=0 > >> >> auto.create.topics.enable=true > >> >> delete.topic.enable=true > >> >> > >> >> listeners=PLAINTEXT://kafka1.example.com:9092<http://kafka1.example.com:9092> > >> >> <http://kafka-dev1.example.com:9092/>,SSL://kafka1.example.com:9093<http://kafka1.example.com:9093> > >> >> <http://kafka-dev1.example.com:9093/> > >> >> host.name<http://host.name>=kafka1.example.com<http://kafka1.example.com> > >> >> <http://kafka-dev1.example.com/> > >> >> > >> >> > >> >> > >> >> ssl.keystore.location=/var/private/kafka1.keystore.jks > >> >> ssl.keystore.password=12345678 > >> >> ssl.key.password=12345678 > >> >> > >> >> ssl.truststore.location=/var/private/kafka1.truststore.jks > >> >> ssl.truststore.password=12345678 > >> >> > >> >> ssl.client.auth=required > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 > >> >> ssl.keystore.type=JKS > >> >> ssl.truststore.type=JKS > >> >> > >> >> authorizer.class.name<http://authorizer.class.name>=kafka.se<http://kafka.se>curity.auth.SimpleAclAuthorizer > >> >> ------------------------------------------------------------ > >> >> ------------------------------------------------ > >> >> > >> >> > >> >> > >> >> Here is producer Config(producer.properties) > >> >> ------------------------------------------------------------ > >> >> ------------------------------------------------ > >> >> security.protocol=SSL > >> >> ssl.truststore.location=/var/private/kafka2.truststore.jks > >> >> ssl.truststore.password=12345678 > >> >> > >> >> ssl.keystore.location=/var/private/kafka2.keystore.jks > >> >> ssl.keystore.password=12345678 > >> >> ssl.key.password=12345678 > >> >> > >> >> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 > >> >> ssl.truststore.type=JKS > >> >> ssl.keystore.type=JKS > >> >> > >> >> ------------------------------------------------------------ > >> >> ------------------------------------------------ > >> >> > >> >> > >> >> Raqhav > >> >> > >> > > >> > > >> > >> > >> -- > >> Raghav > >> > > > > > > > -- > Raghav > -- Raghav -- Raghav The information contained in this e-mail, and any attachment, is confidential and is intended solely for the use of the intended recipient. Access, copying or re-use of the e-mail or any attachment, or any information contained therein, by any other person is not authorized. If you are not the intended recipient please return the e-mail to the sender and delete it from your computer. Although we attempt to sweep e-mail and attachments for viruses, we do not guarantee that either are virus-free and accept no liability for any damage sustained as a result of viruses. Please refer to http://disclaimer.bnymellon.com/eu.htm for certain disclosures relating to European legal entities.
